Before you go mad, I know that you're not supposed to build black boxes, and I agree, so read to the end to get my point.
What is logging?
What most of us as developers do, is console.log() and if you're more experienced you'll be doing some file logging and experts usually using cloud logging, I may be wrong but that's at least what I know. But hey, I'm not here to discuss which method is better, I'm here to say that logging as we (most of us) do is WRONG.
So, again. What is logging?
In computing, a log file is a file that records either events that occur in an operating system or other software runs,[1] or messages between different users of communication software. Logging is the act of keeping a log. In the simplest case, messages are written to a single log file. -- Wikipedia --
Okay, that seems simple, to log, is to save everything happening at some point of time, and that's exactly why I'm saying that logging is bullshit.
Let's be more clear, here's a snippet of an actual logfile from a project I'm working on.
[2019-07-28 05:39:10 AM] INFO: HTTP GET /categories/5d3c5837da5e3f07805c805e {"res":{"statusCode":503},"req":{"url":"/categories/5d3c5837da5e3f07805c805e","headers":{"content-type":"application/json","user-agent":"PostmanRuntime/7.15.2","accept":"*/*","cache-control":"no-cache","postman-token":"8d7392b8-f6bc-4d8b-8d03-e2449a67e8a6","host":"localhost:8080","cookie":"SESSION=NGRjYzUxNTAtZmRjZC00NTA0LWJjNzktNTFkMGIxODYxNmU2","accept-encoding":"gzip, deflate","connection":"keep-alive"},"method":"GET","httpVersion":"1.1","originalUrl":"/categories/5d3c5837da5e3f07805c805e","query":{}},"responseTime":16}
[2019-07-28 05:39:10 AM] ERROR: undefined
[2019-07-28 05:45:16 AM] INFO: service running at *:8080
[2019-07-28 05:45:30 AM] INFO: HTTP GET /categories/5d3c5837da5e3f07805c805e {"res":{"statusCode":503},"req":{"url":"/categories/5d3c5837da5e3f07805c805e","headers":{"content-type":"application/json","user-agent":"PostmanRuntime/7.15.2","accept":"*/*","cache-control":"no-cache","postman-token":"ad0e2e13-0608-436e-aea6-a40d03b40350","host":"localhost:8080","cookie":"SESSION=NGRjYzUxNTAtZmRjZC00NTA0LWJjNzktNTFkMGIxODYxNmU2","accept-encoding":"gzip, deflate","connection":"keep-alive"},"method":"GET","httpVersion":"1.1","originalUrl":"/categories/5d3c5837da5e3f07805c805e","query":{}},"responseTime":17}
[2019-07-28 02:40:04 PM] ERROR: Cannot create namespace beaux_local.categories in multi-document transaction.
MongoError: Cannot create namespace beaux_local.categories in multi-document transaction.
at Connection.<anonymous> (E:\WorkSpace\Projects\Beaux\Server\Rest\node_modules\mongodb-core\lib\connection\pool.js:443:61)
at Connection.emit (events.js:182:13)
at Connection.EventEmitter.emit (domain.js:441:20)
at processMessage (E:\WorkSpace\Projects\Beaux\Server\Rest\node_modules\mongodb-core\lib\connection\connection.js:364:10)
at Socket.<anonymous> (E:\WorkSpace\Projects\Beaux\Server\Rest\node_modules\mongodb-core\lib\connection\connection.js:533:15)
at Socket.emit (events.js:182:13)
at Socket.EventEmitter.emit (domain.js:441:20)
at addChunk (_stream_readable.js:283:12)
at readableAddChunk (_stream_readable.js:264:11)
at Socket.Readable.push (_stream_readable.js:219:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
I'm pretty sure if you're a NodeJs developer, you'll be familiar with this snippet, and it may seem useful for you too, but tell me, what info can you gather from logfile like this? what if this file has about 100k lines or more?
At the end of the day to extract something useful out of these log files, you'll need someone to parse it, or you'll write some piece of software to parse it for you, and that's exactly my point.
So, What I wanna say?
What I want to say is that why we are not logging something parsed and useful from the beginning? actually I have a mindset of not doing what I've told is right but instead, I do what I see is right. So I decided to write a piece of software that extracts the meaning from my logs from the very beginning and save this info parsed, formatted and clean on files that I can read easily instead of these .log files.
Examples are good
Without further ado, here's what I found a useful logfile.
---
level: ERROR
timestamp: '2022-08-07 15:12:22'
message: 'Something wrong'
stack:
- file: 'test.js'
method: Object.<anonymous>
position:
line: 5
column: 59
- file: 'internal/modules/cjs/loader.js'
method: Module._compile
position:
line: 689
column: 30
- file: 'internal/modules/cjs/loader.js'
method: Object.Module._extensions..js
position:
line: 700
column: 10
# ...
Maybe in XML format
<Report>
<Level>ERROR</Level>
<Timestamp>2021-11-31 16:02:00</Timestamp>
<Message>Something wrong</Message>
<Stack>
<Item>
<File>internal/bootstrap/node.js</File>
<Method>bootstrapNodeJSCore</Method>
<Position>
<Line>689</Line>
<Column>30</Column>
</Position>
</Item>
<!-- ... -->
</Stack>
</Report>
Or even JSON
{
"level": "ERROR",
"timestamp": "2022-08-07 15:12:22",
"message": "Something wrong",
"stack": [
{
"file": "test.js",
"method": "Object.<anonymous>",
"position": {
"line": 5,
"column": 59
}
},
{
"file": "internal/modules/cjs/loader.js",
"method": "Module._compile",
"position": {
"line": 689,
"column": 30
}
},
{
"file": "internal/modules/cjs/loader.js",
"method": "Object.Module._extensions..js",
"position": {
"line": 700,
"column": 10
}
}
]
}
Conclusion
I wrote a piece of software called Project "Reporter" aiming to provide a way to help me and other developers to have the best possible experience while working with logs and get the most out of logging or "reporting". For now, it's available for NodeJs and has the following features:
- File Logging
- Cloud Logging
- Console Logging
- Error stack parsing
- JSON based logs
- Yaml based logs
- XML based logs
- Environment based logging
and many more...
And I'm willing to support more languages as well.
If you found it useful, give it a star to keep me motivated ⭐
Contributors are welcome ❤️