feat: add get_all_transactions() method and document API pagination limitations

This commit introduces several improvements for transaction handling:

- Add get_all_transactions() method to fetch up to 500 transactions in one call (works around API's paging-first=0-only limitation)
- Document comdirect API pagination restrictions (paging-first only accepts 0, max 500 transactions via paging-count)
- Add comprehensive strategies for fetching >500 transactions using date filtering and local storage
- Implement token persistence in examples to avoid repeated Push-TAN authentication
- Add remittance_lines property for parsing multi-line remittanceInfo fields
- Update README with pagination limitations, PyPI installation instructions, and advanced usage patterns
- Add COMDIRECT_API.md documenting transaction endpoint pagination behavior

Author: mcdax

COMDIRECT_API.md

@@ -178,7 +178,7 @@ Feature: Comdirect API Client Library
     And an account with accountId exists
     When the user requests transactions for the accountId
     Then the library should check token expiry before request
-    And the library should log "DEBUG: Fetching transactions for account {accountId}"
+    And the library should log "DEBUG: Fetching ALL transactions for account {accountId}"
     And the library should GET /api/banking/v1/accounts/{accountId}/transactions
     And the library should include Authorization header with bearer token
     And the library should include x-http-request-info header
@@ -192,53 +192,48 @@ Feature: Comdirect API Client Library
     And an account with accountId exists
     When the user requests transactions with transactionDirection="DEBIT"
     Then the library should add query parameter "transactionDirection=DEBIT"
-    And the library should log "DEBUG: Fetching transactions (direction: DEBIT)"
+    And the library should log "DEBUG: Fetching ALL transactions for account {accountId} (direction: DEBIT)"
     And the library should return only debit transactions
 
-  Scenario: Retrieve transactions with pagination
-    Given the library is authenticated with valid tokens
-    And an account with accountId exists
-    When the user requests transactions with paging_first=10
-    Then the library should add query parameter "paging-first=10"
-    And the library should log "DEBUG: Fetching transactions (starting at: 10)"
-    And the library should return transactions starting from index 10
-
   Scenario: Retrieve transactions with booking status filter
     Given the library is authenticated with valid tokens
     And an account with accountId exists
     When the user requests transactions with transactionState="BOOKED"
     Then the library should add query parameter "transactionState=BOOKED"
-    And the library should log "DEBUG: Fetching BOOKED transactions"
+    And the library should log "DEBUG: Fetching ALL transactions for account {accountId} (state: BOOKED)"
     And the library should return only booked transactions
 
-  Scenario: Parse transaction response into typed structures
-    Given the library receives a valid transactions response
-    When the library parses the response
-    Then each transaction should be converted to a Transaction object
-    And each Transaction should have a bookingStatus property of type str
-    And each Transaction should have a bookingDate property of type Optional[date]
-    And each Transaction should have an amount property of type Optional[AmountValue]
-    And each Transaction should have optional remitter property of type Optional[AccountInformation]
-    And each Transaction should have optional creditor property of type Optional[AccountInformation]
-    And each Transaction should have a reference property of type str
-    And each Transaction should have a transactionType property of type Optional[EnumText]
-    And each Transaction should have a remittanceInfo property of type Optional[str]
-    And negative amounts should indicate outgoing transactions
-    And positive amounts should indicate incoming transactions
-    And the library should log "DEBUG: Parsed {count} transaction objects"
-
-  Scenario: Handle transactions with null optional fields gracefully
-    Given the library receives a transactions response with null fields
-    And some transactions have null amount
-    And some transactions have null transactionType
-    And some transactions have null remittanceInfo
-    And some transactions have null bookingDate
-    And some AccountInformation objects have null iban
-    When the library parses the response
-    Then the library should not raise an exception
-    And Transaction objects should be created with None values for null fields
-    And the library should safely handle from_dict() calls on null nested objects
-    And the library should log "DEBUG: Parsed {count} transaction objects"
+   Scenario: Parse transaction response into typed structures
+     Given the library receives a valid transactions response
+     When the library parses the response
+     Then each transaction should be converted to a Transaction object
+     And each Transaction should have a bookingStatus property of type str
+     And each Transaction should have a bookingDate property of type Optional[date]
+     And each Transaction should have an amount property of type Optional[AmountValue]
+     And each Transaction should have optional remitter property of type Optional[AccountInformation]
+     And each Transaction should have optional creditor property of type Optional[AccountInformation]
+     And each Transaction should have a reference property of type str
+     And each Transaction should have a transactionType property of type Optional[EnumText]
+     And each Transaction should have a remittanceLines field of type list[str]
+     And each Transaction should expose a remittance_lines property returning the same list
+     And negative amounts should indicate outgoing transactions
+     And positive amounts should indicate incoming transactions
+     And the library should log "DEBUG: Parsed {count} transaction objects"
+
+
+   Scenario: Handle transactions with null optional fields gracefully
+     Given the library receives a transactions response with null fields
+     And some transactions have null amount
+     And some transactions have null transactionType
+     And some transactions have null bookingDate
+     And some AccountInformation objects have null iban
+     When the library parses the response
+     Then the library should not raise an exception
+     And Transaction objects should be created with None values for null fields
+     And Transaction objects should have an empty remittanceLines list when remittanceInfo is missing or empty
+     And the library should safely handle from_dict() calls on null nested objects
+     And the library should log "DEBUG: Parsed {count} transaction objects"
+
 
   # ============================================================================
   # Error Handling and Logging
@@ -551,3 +546,32 @@ Feature: Comdirect API Client Library
     And transaction.debtor should be populated from "debtor"
     And "deptor" should be ignored
 
+   # ============================================================================
+   # New: remittance line marker handling
+   # ============================================================================
+
+   Scenario: Parse remittanceInfo with numbered line prefixes into remittanceLines
+     Given the library receives a valid transactions response
+     And some transactions have remittanceInfo values with numbered line prefixes
+     When the library parses the response
+     Then each Transaction should have a remittanceLines field of type list[str]
+     And each Transaction should expose a remittance_lines property returning the same list
+     And the library should detect numbered line markers (01, 02, 03, ... up to 99)
+     And the library should support both short test-format remittanceInfo strings
+     And the library should support long fixed-width remittanceInfo strings from the real API
+     And for long-format strings, the library should detect markers using approximate spacing between markers
+     And the library should extract the content after each marker as a logical line
+     And the library should populate remittanceLines with one entry per logical line, in order
+     And example "01AA02 N84G BFT2 Y5KY                02End-to-End-Ref.:                   03nicht angegeben                    " should produce remittanceLines:
+       | AA02 N84G BFT2 Y5KY
+       | End-to-End-Ref.:
+       | nicht angegeben
+     And example "01Storno Echtzeitüberweisung         02AA02 N84G BFT2 Y5KY                03Rückgabegrund:                     04Auf Veranlassung der Bank          " should produce remittanceLines:
+       | Storno Echtzeitüberweisung
+       | AA02 N84G BFT2 Y5KY
+       | Rückgabegrund:
+       | Auf Veranlassung der Bank
+     And the library should strip trailing whitespace from each line
+     And the library should skip empty lines after stripping
+
+

README.md

@@ -762,22 +762,24 @@ async def get_transactions(
         account_id: str,
         transaction_state: Optional[str] = None,
         transaction_direction: Optional[str] = None,
-        paging_first: Optional[int] = None,
         with_attributes: bool = True,
         without_attributes: Optional[str] = None,
     ) -> list[Transaction]:
-        """Retrieve transactions for a specific account.
+        """Retrieve **all available** transactions for a specific account.
+
+        This method now uses the API's maximum page size to fetch up to
+        500 transactions in a single call (API limit). For accounts with more
+        than 500 transactions, only the most recent 500 will be returned.
 
         Args:
             account_id: Account UUID (from AccountBalance.accountId)
             transaction_state: Optional filter: "BOOKED", "NOTBOOKED", or "BOTH" (default: "BOTH")
             transaction_direction: Optional filter: "CREDIT", "DEBIT", or "CREDIT_AND_DEBIT" (default: "CREDIT_AND_DEBIT")
-            paging_first: Optional index of first transaction for pagination (default: 0)
             with_attributes: Include account details in response (default: True)
             without_attributes: Comma-separated list of attributes to exclude (optional)
 
         Returns:
-            List of Transaction objects
+            List of Transaction objects (up to 500 per call)
 
         Raises:
             TokenExpiredError: If authentication token is expired
@@ -788,14 +790,12 @@ async def get_transactions(
         """
         await self._ensure_authenticated()
 
-        # Build query parameters
-        params: dict[str, str] = {}
+        # Build query parameters with maximum page size
+        params: dict[str, str] = {"paging-count": "500"}
         if transaction_state:
             params["transactionState"] = transaction_state
         if transaction_direction:
             params["transactionDirection"] = transaction_direction
-        if paging_first is not None:
-            params["paging-first"] = str(paging_first)
         if not with_attributes:
             params["without-attr"] = "account"
         if without_attributes:
@@ -804,13 +804,11 @@ async def get_transactions(
             else:
                 params["without-attr"] = without_attributes
 
-        log_msg = f"Fetching transactions for account {account_id[:8]}..."
+        log_msg = f"Fetching ALL transactions for account {account_id[:8]}... (max: {params['paging-count']})"
         if transaction_direction:
             log_msg += f" (direction: {transaction_direction})"
         if transaction_state:
             log_msg += f" (state: {transaction_state})"
-        if paging_first is not None:
-            log_msg += f" (starting at: {paging_first})"
         logger.debug(log_msg)
 
         try:
@@ -821,7 +819,7 @@ async def get_transactions(
                     "Accept": "application/json",
                     "x-http-request-info": self._get_request_info_header(),
                 },
-                params=params if params else None,
+                params=params,
             )
 
             if response.status_code == 401:
@@ -835,7 +833,6 @@ async def get_transactions(
                     account_id,
                     transaction_state,
                     transaction_direction,
-                    paging_first,
                     with_attributes,
                     without_attributes,
                 )
@@ -856,9 +853,18 @@ async def get_transactions(
             data = response.json()
 
             transactions = [Transaction.from_dict(item) for item in data["values"]]
-            logger.info(f"Retrieved {len(transactions)} transactions")
+            logger.info(
+                f"Retrieved {len(transactions)} transactions for account {account_id[:8]}..."
+            )
             logger.debug(f"Parsed {len(transactions)} transaction objects")
 
+            # Log pagination info for debugging
+            if "paging" in data:
+                paging_info = data["paging"]
+                logger.debug(
+                    f"Pagination: index={paging_info.get('index')}, matches={paging_info.get('matches')}"
+                )
+
             return transactions
 
         except httpx.TimeoutException as e:

comdirect_api.feature

@@ -1,6 +1,6 @@
 """Data models for the Comdirect API client."""
 
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from datetime import date
 from decimal import Decimal
 from typing import Any, Optional
@@ -100,6 +100,108 @@ def from_dict(cls, data: dict[str, Any]) -> "AccountBalance":
         )
 
 
+def _parse_remittance_info(remittance: Optional[str]) -> list[str]:
+    """Parse Comdirect remittanceInfo string into logical lines.
+
+    The Comdirect API encodes line breaks in remittanceInfo by prefixing each
+    logical line with a two-digit sequence number ("01", "02", ...).
+
+    Two formats are supported:
+    1. Long format (real API data): Markers at ~37-char intervals
+    2. Short format (test data): Markers with variable/close spacing
+
+    This function adapts to detect which format is used and extracts lines accordingly.
+    """
+
+    if not remittance:
+        return []
+
+    text = remittance.strip()
+    if not text:
+        return []
+
+    length = len(text)
+
+    # Find the first marker (01)
+    first_pos = -1
+    if length >= 2 and text[0:2] == "01":
+        first_pos = 0
+    else:
+        for i in range(length - 2):
+            if text[i].isspace() and text[i + 1 : i + 3] == "01":
+                first_pos = i + 1
+                break
+
+    if first_pos == -1:
+        # No valid starting marker – treat entire string as single line
+        return [text]
+
+    # Detect format based on total length and marker spacing
+    # Long format typically has 37-char intervals and total length > 100
+    # Short format has variable/close spacing and total length < 100
+    is_long_format = length > 100
+
+    marker_positions: list[int] = [first_pos]
+    expected_marker = 2
+
+    if is_long_format:
+        # Long format: use spacing heuristic with tolerance
+        expected_pos = first_pos + 37
+        tolerance = 15
+
+        while expected_marker <= 99:
+            found = False
+            search_start = max(marker_positions[-1] + 20, expected_pos - tolerance)
+            search_end = min(length - 1, expected_pos + tolerance)
+
+            for pos in range(search_start, search_end):
+                if pos + 1 < length and text[pos].isdigit() and text[pos + 1].isdigit():
+                    marker_value = int(text[pos : pos + 2])
+                    if marker_value == expected_marker:
+                        marker_positions.append(pos)
+                        expected_pos = pos + 37
+                        expected_marker += 1
+                        found = True
+                        break
+
+            if not found:
+                break
+    else:
+        # Short format: scan for markers that appear after whitespace
+        # This avoids false positives in timestamps like "2020-01-03T20:07:16"
+        while expected_marker <= 99:
+            found = False
+            search_start = marker_positions[-1] + 2
+
+            for pos in range(search_start, length - 1):
+                # Marker must be after whitespace (not in middle of numbers/text)
+                if (
+                    text[pos].isdigit()
+                    and text[pos + 1].isdigit()
+                    and (pos == 0 or text[pos - 1].isspace())
+                ):
+                    marker_value = int(text[pos : pos + 2])
+                    if marker_value == expected_marker:
+                        marker_positions.append(pos)
+                        expected_marker += 1
+                        found = True
+                        break
+
+            if not found:
+                break
+
+    # Extract lines between markers
+    lines: list[str] = []
+    for idx, pos in enumerate(marker_positions):
+        start = pos + 2  # skip the two-digit marker
+        end = marker_positions[idx + 1] if idx + 1 < len(marker_positions) else length
+        line = text[start:end].strip()
+        if line:
+            lines.append(line)
+
+    return lines if lines else [text]
+
+
 @dataclass
 class Transaction:
     """Bank account transaction.
@@ -116,8 +218,9 @@ class Transaction:
     newTransaction: bool
     amount: Optional[AmountValue] = None
     transactionType: Optional[EnumText] = None
-    remittanceInfo: Optional[str] = None
+    remittanceLines: list[str] = field(default_factory=list)
     bookingDate: Optional[date] = None
+
     remitter: Optional[AccountInformation] = None
     debtor: Optional[AccountInformation] = None
     creditor: Optional[AccountInformation] = None
@@ -141,13 +244,15 @@ def from_dict(cls, data: dict[str, Any]) -> "Transaction":
         if data.get("transactionType"):
             transaction_type = EnumText.from_dict(data["transactionType"])
 
+        remittance_lines = _parse_remittance_info(data.get("remittanceInfo"))
+
         return cls(
             bookingStatus=data["bookingStatus"],
             amount=amount,
             reference=data["reference"],
             valutaDate=data["valutaDate"],
             transactionType=transaction_type,
-            remittanceInfo=data.get("remittanceInfo"),
+            remittanceLines=remittance_lines,
             newTransaction=data["newTransaction"],
             bookingDate=booking_date,
             remitter=(
@@ -165,3 +270,14 @@ def from_dict(cls, data: dict[str, Any]) -> "Transaction":
             directDebitCreditorId=data.get("directDebitCreditorId"),
             directDebitMandateId=data.get("directDebitMandateId"),
         )
+
+    @property
+    def remittance_lines(self) -> list[str]:
+        """Return remittance lines for this transaction.
+
+        The raw `remittanceInfo` string from the API is parsed once in
+        `from_dict` into `remittanceLines`. This property simply returns
+        that list.
+        """
+
+        return self.remittanceLines