What is Logging?
Logging (also known as LaaS, logging as a service) is a service that is used to collect and store logs of virtual machines and baremetal servers. It doesn't matter whether you use a G-Core Labs server or your own one: Logging will collate logs from any devices. You can work with logs via OpenSearch Dashboards.
How Logging works
- We allocate storage space for your logs.
- You install a log shipper on your servers — the tool collects logs and forwards them to our storage. For example, you can install Fluent Bit or Filebeat log shipper.
- You configure the log shipper: specify service(s) whose logs need to be exported to the storage. If you haven’t worked with a log shipper before, the following instructions may help: Configure Fluent Bit, Configure Filebeat.
- Logs from your machine are automatically exported to the storage.
- You work with the logs of your resources via OpenSearch Dashboards that we provide.
How Logging can be useful
Logging collects logs from different services in one dashboard. To find a cause of an error or to collect data for analytics, you do not have to open logs of each service or machine manually — all data is available in OpenSearch Dashboards. You can apply filters to see the required logs.
Logs are still available even if their source is destroyed — for example, a server is broken, or a Kubernetes pod is deleted. By opening the Logging tab, you can see what happened to a machine before the breakdown or to an object before the deletion.
When Logging may be useful
With Logging, you can see how the system has changed. Below are the examples of four cases when the feature may come in handy.
Quicker detection and fix of errors in a cluster. You enter an error message into the OpenSearch Dashboards search bar, identify when and which server experienced an issue, and fix it.
It turned out that data on your server was stolen last month. You find logs from a month ago, see an account that signed in to the storage, and detect the culprit.
Server connectivity check. You open logs of two servers for the same time interval and see whether all the data has reached an end-user.
Data speed measurement. You open logs and compare the time when one server sent the data with the time when another one received it.
How we store logs
Logs from your machines are exported to Kafka servers, which then send them to OpenSearch servers for permanent storage. You read the logs from the OpenSearch servers via OpenSearch Dashboards.
We use Kafka servers as a buffer for better reliability. Even if an OpenSearch server fails to receive logs, they will be saved in Kafka and sent to OpenSearch as soon as possible. We have built such a system to ensure the safety of your data.