How to set up a CrowdSec multi-server installation

How to set up a CrowdSec multi-server installation

Introduction

CrowdSec is an open-source & collaborative security solution built to secure Internet-exposed Linux services, servers, containers, or virtual machines with a server-side agent. It is a modernized version of Fail2ban which was a great source of inspiration to the project founders.

CrowdSec is free (under an MIT License) and its source code available on GitHub. The solution is leveraging a log-based IP behavior analysis engine to detect attacks. When the CrowdSec agent detects any aggression, it offers different types of remediation to deal with the IP behind it (access prohibition, captcha, 2FA authentication etc.). The report is curated by the platform and, if legitimate, shared across the CrowdSec community so users can also protect their assets from this IP address.

A few months ago, we added some interesting features to CrowdSec when releasing v1.0.x. One of the most exciting ones is the ability of the CrowdSec agent to act as an HTTP rest API to collect signals from other CrowdSec agents. Thus, it is the responsibility of this special agent to store and share the collected signals. We will call this special agent the LAPI server from now on.

Another worth noting feature, is that mitigation no longer has to take place on the same server as detection. Mitigation is done using bouncers. Bouncers rely on the HTTP REST API served by the LAPI server.

Goals

In this article we’ll describe how to deploy CrowdSec in a multi-server setup with one server sharing signal.

CrowdSec Goals Infographic

Both server-2 and server-3 are meant to host services. You can take a look on our Hub to know which services CrowdSec can help you secure. Last but not least, server-1 is meant to host the following local services:

  • the local API needed by bouncers

  • the database fed by both the three local CrowdSec agents and the online CrowdSec blocklist service. As server-1 is serving the local API, we will call it the LAPI server.

We choose to use a postgresql backend for CrowdSec database in order to allow high availability. This topic will be covered in future posts. If you are ok with no high availability, you can skip step 2.

Develop a Linux command-line Tool to Track and Plot Covid-19 Stats

Develop a Linux command-line Tool to Track and Plot Covid-19 Stats

It’s been over a year and we are still fighting with the pandemic at almost every aspect of our life. Thanks to technology, we have various tools and mechanisms to track Covid-19 related metrics which help us make informed decisions. This introductory-level tutorial discusses developing one such tool at just Linux command-line, from scratch.

We will start with introducing the most important parts of the tool – the APIs and the commands. We will be using 2 APIs for our tool - COVID19 API and Quickchart API and 2 key commands – curl and jq. In simple terms, curl command is used for data transfer and jq command to process JSON data.

The complete tool can be broken down into 2 keys steps:

1. Fetching (GET request) data from the COVID19 API and piping the JSON output to jq so as to process out only global data (or similarly, country specific data).

$ curl -s --location --request GET 'https://api.covid19api.com/summary' | jq -r '.Global'

{

  "NewConfirmed": 561661,

  "TotalConfirmed": 136069313,

  "NewDeaths": 8077,

  "TotalDeaths": 2937292,

  "NewRecovered": 487901,

  "TotalRecovered": 77585186,

  "Date": "2021-04-13T02:28:22.158Z"

}

2. Storing the output of step 1 in variables and calling the Quickchart API using those variables, to plot a chart. Subsequently piping the JSON output to jq so as to filter only the link to our chart.

$ curl -s -X POST \

       -H 'Content-Type: application/json' \

       -d '{"chart": {"type": "bar", "data": {"labels": ["NewConfirmed ('''${newConf}''')", "TotalConfirmed ('''${totConf}''')", "NewDeaths ('''${newDeath}''')", "TotalDeaths ('''${totDeath}''')", "NewRecovered ('''${newRecover}''')", "TotalRecovered ('''${totRecover}''')"], "datasets": [{"label": "Global Covid-19 Stats ('''${datetime}''')", "data": ['''${newConf}''', '''${totConf}''', '''${newDeath}''', '''${totDeath}''', '''${newRecover}''', '''${totRecover}''']}]}}}' \

       https://quickchart.io/chart/create | jq -r '.url'

https://quickchart.io/chart/render/zf-be27ef29-4495-4e9a-9180-dbf76f485eaf

That’s it! Now we have our data plotted out in a chart:

LJ Global-Stats-Track-And-Plot-Covid19-Stats