Add streaming API for zero-allocation event based parsing

diffstorm · diffstorm · commit 0a3bc7eb2d62 · 2025-05-24T18:17:57.000+01:00
diff --git a/README.md b/README.md
@@ -7,18 +7,137 @@
 ![GitHub Stars](https://img.shields.io/github/stars/diffstorm/ini_parser?style=social)
 ![Platforms](https://img.shields.io/badge/Platform-Linux%20%7C%20Windows%20%7C%20macOS-lightgrey)
 
-A lightweight, single-header, speed and safety focused INI file parsing library written in C with C++ compatibility. Designed for simplicity and portability, this parser provides a low-footprint solution to decode INI format.
+A lightweight, single-header INI file parsing library written in C, with C++ compatibility. Focused on speed and safety, it offers a low-footprint, portable solution for decoding INI files. The library provides two APIs to suit different use cases: fast random access and memory-efficient streaming.
 
 ## Features
-
+- **Dual Parsing Modes**
+  - **Context API**: Builds searchable structure (O(1) lookups)
+  - **Streaming API**: Zero-allocation event-based parsing
+- **Memory Safe**: Automatic cleanup with context API
+- **Unicode Ready**: Full UTF-8 support
 - **Robust Parsing**: Handles sections, key-value pairs, comments, and empty lines
 - **Memory Safe**: Automatic cleanup of allocated resources
 - **Whitespace Handling**: Trims leading/trailing spaces in sections and values
 - **Thread Safe**: Context-based design enables multi-instance usage
 - **Cross-Platform**: C99/C++11 compatible with no external dependencies
-- **Configurable**: Adjustable line length via `INI_MAX_LINE_LENGTH`
+- **Configurable**: Adjust line length (`INI_MAX_LINE_LENGTH`) and case sensitivity
+
+## API Comparison
+
+| Feature                | Context API (`ini_initialize`) | Streaming API (`ini_parse_stream`) |
+|------------------------|--------------------------------|------------------------------------|
+| **Memory Usage**       | Builds full structure          | Constant memory                    |
+| **Lookup Speed**       | O(1) after init                | N/A (sequential)                   |
+| **Best For**           | Small files, random access     | Large files (>1GB), embedded       |
+| **Error Handling**     | Post-parse validation          | Immediate feedback                 |
+| **Thread Safety**      | Per-context isolation          | Handler must be reentrant          |
+| **Max File Size**      | Limited by memory              | Unlimited                          |
+| **Needs init?**        | Yes (`ini_initialize`          | No                                 |
+| **Needs cleanup?**     | Yes (`ini_cleanup`)            | No                                 |
+
+**Why No Initialization/Cleanup Needed for Streaming:**
+- Processes lines directly from input buffer
+- Doesn't build any persistent data structures
+- All temporary buffers are stack-allocated
+- User handles state via `userdata` parameter
+
+**Memory Diagram:**
+```
+Context API:              Streaming API:
++---------------+         +-----------------+
+| ini_context_t |         | Input Buffer    |
+| - sections    |         +-----------------+
+| - keyValues   |         | Line Buffer     | (stack)
++---------------+         +-----------------+
+| Allocations   |         | Userdata        |
++---------------+         +-----------------+
+                          | No persistent   |
+                          | state           |
+                          +-----------------+
+```
+
+## Choosing an API
+
+**Use Context API When:**
+- You need random access to values
+- Working with small to medium configs
+- Frequent lookups needed after parsing
+
+**Use Streaming API When:**
+- Processing large files (>100MB)
+- Memory-constrained environments
+- Simple validation/transformation needed
+- Direct loading to application structures
+
+## Streaming Parsing API
+
+### Core Components
+
+```c
+typedef enum {
+    INI_EVENT_SECTION,    // New section found
+    INI_EVENT_KEY_VALUE,  // Key-value pair parsed
+    INI_EVENT_COMMENT,    // Comment line detected
+    INI_EVENT_ERROR       // Parse error encountered
+} ini_eventtype_t;
+
+typedef bool (*ini_handler)(
+    ini_eventtype_t type,  // Event type
+    const char* section,   // Current section (if applicable)
+    const char* key,       // Key name (for key-value events)
+    const char* value,     // Value or comment content
+    void* userdata         // Custom user context
+);
+```
+
+### Functions
+
+#### `bool ini_parse_stream(const char* content, size_t length, ini_handler handler, void* userdata)`
+Processes INI content line-by-line
+- `content`: Input string (not modified)
+- `length`: Input length in bytes
+- `handler`: Callback for parsed events
+- `userdata`: Custom context pointer
+- **Returns**: `false` if handler aborted parsing
 
-## Components
+### Usage Example
+
+```c
+typedef struct {
+    int sections;
+    int keys;
+} ParserStats;
+
+bool stats_handler(ini_eventtype_t type, 
+                  const char* section,
+                  const char* key,
+                  const char* value,
+                  void* userdata) {
+    ParserStats* stats = (ParserStats*)userdata;
+    
+    switch(type) {
+    case INI_EVENT_SECTION:
+        stats->sections++;
+        break;
+    case INI_EVENT_KEY_VALUE:
+        stats->keys++;
+        printf("[%s] %s = %s\n", section, key, value);
+        break;
+    case INI_EVENT_ERROR:
+        fprintf(stderr, "Error in: %s\n", value);
+        return false; // Abort parsing
+    }
+    return true;
+}
+
+void process_large_ini(const char* content, size_t length) {
+    ParserStats stats = {0};
+    ini_parse_stream(content, length, stats_handler, &stats);
+    printf("Parsed %d sections with %d keys\n", stats.sections, stats.keys);
+}
+```
+
+## Context API
 
 ### Core Structures
 - **`ini_context_t`**: Manages parser state and stored sections
@@ -74,9 +193,35 @@ Retrieves value for specified section/key
 - `maxLen`: Maximum buffer size
 - **Returns**: `true` if value found and copied
 
-### Configuration Macros
+## Usage Example
+
+```c
+#include "ini_parser.h"
+#include <stdio.h>
+
+int main() {
+    const char *config = 
+        "[network]\n"
+        "host = 127.0.0.1\n"
+        "port = 8080\n";
+    
+    ini_context_t ctx = {0};
+    if (ini_initialize(&ctx, config, strlen(config))) {
+        char host[256], port[16];
+        if (ini_getValue(&ctx, "network", "host", host, sizeof(host)) &&
+            ini_getValue(&ctx, "network", "port", port, sizeof(port))) {
+            printf("Connecting to %s:%s\n", host, port);
+        }
+        ini_cleanup(&ctx);
+    }
+    return 0;
+}
+```
+
+## Configuration Macros
 - `INI_MAX_LINE_LENGTH`: Maximum allowed line length (default: 256)
 - `INI_PARSER_IMPLEMENTATION`: Define to enable implementation inclusion
+- `INI_ENABLE_CASE_SENSITIVITY`: Enables case sensitivity for sections, keys and values.
 
 ## Error Handling
 The parser provides implicit error checking through boolean return values. Common failure scenarios:
@@ -100,35 +245,20 @@ cmake --build .
 ctest --output-on-failure
 ```
 
-## Usage Example
+## Performance Tips
 
-```c
-#include "ini_parser.h"
-#include <stdio.h>
+1. **Prefer Streaming API** for files >100MB
+2. **Reuse Contexts** when possible
+3. **Set INI_MAX_LINE_LENGTH** to match your data
+4. **Avoid Case Sensitivity** unless required (`INI_ENABLE_CASE_SENSITIVITY`)
+5. **Batch Initializations** for multiple small configs
 
-int main() {
-    const char *config = 
-        "[network]\n"
-        "host = 127.0.0.1\n"
-        "port = 8080\n";
-    
-    ini_context_t ctx = {0};
-    if (ini_initialize(&ctx, config, strlen(config))) {
-        char host[256], port[16];
-        if (ini_getValue(&ctx, "network", "host", host, sizeof(host)) &&
-            ini_getValue(&ctx, "network", "port", port, sizeof(port))) {
-            printf("Connecting to %s:%s\n", host, port);
-        }
-        ini_cleanup(&ctx);
-    }
-    return 0;
-}
-```
+## Quality Assurance
 
-## Performance
-- **O(1) Lookups**: Pre-parsed structure enables fast access
-- **Low Memory**: Compact representation with single allocation
-- **Efficient Parsing**: Linear time complexity O(n)
+- **Valgrind Clean**: Zero memory leaks
+- **100% Line Coverage**: Comprehensive test suite
+- **Fuzz Tested**: AFL-integrated validation
+- **SAN Enabled**: Address/UB sanitizer support
 
 ## :snowman: Author
 
diff --git a/ini_parser.h b/ini_parser.h
@@ -61,13 +61,24 @@ typedef struct
     ini_section_t *sections;
 } ini_context_t;
 
+typedef enum
+{
+    INI_EVENT_SECTION,
+    INI_EVENT_KEY_VALUE,
+    INI_EVENT_COMMENT,
+    INI_EVENT_ERROR
+} ini_eventtype_t;
+
+typedef bool (*ini_handler)(ini_eventtype_t type, const char *section, const char *key, const char *value, void *userdata);
+
 bool ini_initialize(ini_context_t *ctx, const char *content, size_t length);
 void ini_cleanup(ini_context_t *ctx);
 bool ini_hasSection(const ini_context_t *ctx, const char *section);
 bool ini_hasKey(const ini_context_t *ctx, const char *section, const char *key);
 bool ini_hasValue(const ini_context_t *ctx, const char *section, const char *key);
 bool ini_getValue(const ini_context_t *ctx, const char *section, const char *key,
                   char *value, size_t maxLen);
+bool ini_parse_stream(const char *content, size_t length, ini_handler handler, void *userdata);
 
 #ifdef __cplusplus
 }
@@ -455,4 +466,90 @@ bool ini_getValue(const ini_context_t *ctx, const char *section, const char *key
     return false;
 }
 
+bool ini_parse_stream(const char *content, size_t length, ini_handler handler, void *userdata)
+{
+    if(!content || !handler)
+    {
+        return false;
+    }
+
+    const char *ptr = content;
+    const char *end = content + length;
+    char line[INI_MAX_LINE_LENGTH];
+    char current_section[INI_MAX_LINE_LENGTH] = "";
+
+    while(ptr < end)
+    {
+        // Extract line
+        const char *line_start = ptr;
+
+        while(ptr < end && *ptr != '\n' && *ptr != '\r')
+        {
+            ptr++;
+        }
+
+        size_t line_len = ptr - line_start;
+
+        // Handle line endings
+        while(ptr < end && (*ptr == '\n' || *ptr == '\r'))
+        {
+            ptr++;
+        }
+
+        // Process line
+        if(line_len > 0)
+        {
+            line_len = line_len < INI_MAX_LINE_LENGTH - 1 ? line_len : INI_MAX_LINE_LENGTH - 1;
+            memcpy(line, line_start, line_len);
+            line[line_len] = '\0';
+            char section[INI_MAX_LINE_LENGTH] = "";
+            char key[INI_MAX_LINE_LENGTH] = "";
+            char value[INI_MAX_LINE_LENGTH] = "";
+            ini_linetype_t type = parseLine(line, section, key, value);
+
+            switch(type)
+            {
+                case INI_LINE_SECTION:
+                    strncpy(current_section, section, INI_MAX_LINE_LENGTH);
+
+                    if(!handler(INI_EVENT_SECTION, current_section, NULL, NULL, userdata))
+                    {
+                        return false;
+                    }
+
+                    break;
+
+                case INI_LINE_KEY_VALUE:
+                    if(!handler(INI_EVENT_KEY_VALUE, current_section, key, value, userdata))
+                    {
+                        return false;
+                    }
+
+                    break;
+
+                case INI_LINE_COMMENT:
+                    if(!handler(INI_EVENT_COMMENT, NULL, NULL, line, userdata))
+                    {
+                        return false;
+                    }
+
+                    break;
+
+                case INI_LINE_INVALID:
+                    if(!handler(INI_EVENT_ERROR, NULL, NULL, line, userdata))
+                    {
+                        return false;
+                    }
+
+                    break;
+
+                default:
+                    break;
+            }
+        }
+    }
+
+    return true;
+}
+
 #endif /* INI_PARSER_IMPLEMENTATION */
diff --git a/ini_parser_tests.cpp b/ini_parser_tests.cpp
