Log Structure (Log Builder)

The Log Builder creates a standardized and structured log payload before sending it to the Data Nadhi Server.
It makes sure logs are consistent, traceable, and easy to process — no matter which language or SDK you're using.

---

Purpose

• Standardize the structure of logs sent from all SDKs.
• Include essential metadata like timestamps, trace IDs, and source location.
• Keep application log format and Data Nadhi transmission format separate — so your log formatter can still be fully custom.

---

Log Payload Schema

A structured log generated by the SDK typically includes the following fields:

Field	Type	Description
message	string	The actual log message generated by the user or application.
trace_id	string	A unique identifier used to trace the flow of a single request or operation across multiple logs.
timestamp	string	UTC timestamp in ISO 8601 format (e.g., 2025-10-23T10:45:00Z).
log_record	object	Metadata about where and how the log was generated (see below).
context	object	Custom structured data provided by the user — useful for adding business or request-specific information.

---

log_record Object

Field	Type	Description
filename	string	Name of the source file where the log originated.
function_name	string	The function or method name where the log was triggered.
level	string	The severity level of the log (e.g., INFO, DEBUG, ERROR).
line_number	number	Line number in the source file.
module_name	string	The module or component name that produced the log.

---

Notes

• The SDK automatically collects log_record info at runtime.
• The log payload format is consistent across all SDKs to keep things compatible with the Data Nadhi backend.
• Your application log formatter can still be different — this structure only defines what gets sent to the Data Nadhi Server.
• The context field is user-defined and can have nested JSON objects.

---

Example Structure

{
  "message": "User login successful",
  "trace_id": "e5f2a0b9-ff12-4a88-9b61-44c203ef11a7",
  "timestamp": "2025-10-23T10:45:00Z",
  "log_record": {
    "filename": "auth_handler.js",
    "function_name": "loginUser",
    "level": "INFO",
    "line_number": 42,
    "module_name": "auth"
  },
  "context": {
    "user_id": "12345",
    "region": "us-east-1"
  }
}

---

---

Trace ID Behavior

The Trace ID is a special context variable used to correlate logs from the same execution flow, request, or transaction.
It lets you group and analyze related logs together — even across distributed systems.

Default Behavior

• If no trace ID exists when a log is created, the SDK automatically generates a new UUID as the trace ID.
• This trace ID stays active for the current execution context and gets automatically attached to all subsequent logs.

Overriding Behavior

• If a trace ID is provided in the log as keyword arg trace_id or as third argument to the log, it overrides the existing trace ID for the current execution scope.
• From that point on, all new logs use the new trace ID automatically, until another override happens or the context resets.

Context Handling

• In languages that support context variables, the SDK stores the trace ID in the current context (thread, coroutine, or request).
• This gives you automatic propagation without having to manually pass the trace ID every time.

Why This Matters

• Enables distributed tracing across microservices.
• Simplifies log correlation for debugging and performance tracking.
• Gives you a consistent trace chain even across asynchronous or multi-threaded operations.

---

log_record Details

The log_record field has metadata about where and how a log was generated.
While the structure is conceptually standardized, the actual implementation differs slightly between languages based on what metadata is available from their native logging modules.

Common Fields

Field	Type	Description
filename	string	The name of the source file where the log originated.
function_name	string	The function or method that emitted the log.
level	string	The severity level (e.g., DEBUG, INFO, ERROR).
line_number	number	Line number of the log statement in the source code.
module_name	string	The logical module or component name.

Implementation Notes

• The log_record object is custom-built per SDK, using the native logging capabilities of that language. But the naming convention stays the same across all SDKs for a particular field.
• The goal is to include as much contextual info as possible — without adding unnecessary overhead.
• Some SDKs might include additional metadata like:
  • Thread or process ID
  • File path
  • Class or package name
  • Runtime environment or service name
• Each SDK keeps the same top-level schema but can extend it for better observability in that ecosystem.

---

Key Notes

• The log payload structure stays consistent across SDKs for interoperability.
• Application log formatters are unaffected — this structure only defines what gets sent to the Data Nadhi Server.
• The context field is flexible and can have any nested JSON structure.
• Trace ID propagation and log_record customization together give you both detailed local debugging and global trace correlation.

---

In short:
The Log Builder makes sure every log — no matter which language or framework you're using — carries a complete, structured, and traceable payload, making cross-service observability simple and consistent.