fix(idempotency): resolve tech debt issues with falsy responses and Redis persistency (#8176)

* fix(idempotency): fix str(None) producing "None" string in Redis persistence

Replace str(item.get(...)) with item.get(...) to avoid storing the
string "None" when a value is missing from Redis hash map.

When data_attr or validation_key_attr is missing from Redis,
item.get() returns None. Wrapping with str() converts it to the
string "None" which is incorrect. Now correctly returns None.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* refactor(idempotency): extract duplicated idempotency key null-check into helper method

Replace 4 identical null-check blocks across save_success, save_inprogress, delete_record, and get_record with a single helper method _get_idempotency_key_or_return_none() to reduce code duplication.

The helper encapsulates the pattern of calling _get_hashed_idempotency_key() and returning None early if the key is None, keeping each method cleaner and easier to read.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* test(idempotency): add unit tests for tech debt fixes in issue #8090

Fix 1 - str(None) in Redis _item_to_data_record:
- Missing data_attr returns None not string "None"
- Existing data_attr returns value correctly
- Demonstrates old bug vs new correct behavior

Fix 2 - _get_idempotency_key_or_return_none helper:
- Returns None when key is None
- Returns key string when key exists
- Correctly used in save_success, save_inprogress,
  delete_record, and get_record (all return None early)

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* fix(idempotency): fix falsy response handling and inconsistent status constant

Fix 1 - Falsy response handling in _get_function_response(): Replace `if response else None` with `if response is not None else None`. So valid falsy return values (0, False, {}, [], "") are correctly   serialized and stored instead of being silently discarded.

Fix 2 - Inconsistent status constant in _process_idempotency():
Replace string literal "INPROGRESS" with STATUS_CONSTANTS["INPROGRESS"] for consistency with the rest of the codebase.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* refactor(idempotency): revert helper method - restore original inline null-check pattern

Per Leandro Damascena's feedback: _get_idempotency_key_or_return_none() helper added
indirection without reducing duplication since the if None: return None check remained in all 4 callers anyway.

Restored original inline 3-line pattern in save_success, save_inprogress, delete_record, and get_record which is clearer and instantly readable.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* test(idempotency): remove standalone test file per maintainer feedback

Tests should be added to existing test files following established patterns, not in new standalone files. The Redis fix will be tested in
_redis/test_redis_layer.py next to the existing test_item_to_datarecord_conversion.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* test(idempotency): add regression test for str(None) fix in _item_to_data_record

Added single test next to existing test_item_to_datarecord_conversion to verify missing Redis attributes return None instead of string "None".

Follows existing test patterns using fixtures instead of MagicMock.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* fix(idempotency): fix ruff formatting - remove trailing whitespace in base.py

Remove trailing whitespace on blank line between is_missing_idempotency_key
and _get_hashed_payload methods to pass ruff format check.

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* fix(idempotency): fix test fixture name and trailing whitespace in test_redis_layer.py

- Fix fixture name from persistence_store_redis to persistence_store_standalone_redis
  to match existing fixture defined in the test file
- Remove trailing whitespace on blank line after test function to pass ruff format check

Part of #8090

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>

* fix: small changes

---------

Signed-off-by: hirenkumar-n-dholariya <hirenkumarnd@gmail.com>
Co-authored-by: Leandro Damascena <lcdama@amazon.pt>

Author: hirenkumar-n-dholariya

aws_lambda_powertools/utilities/idempotency/base.py

@@ -167,7 +167,7 @@ def _process_idempotency(self, is_replay: bool):
             # We give preference to ReturnValuesOnConditionCheckFailure because it is a faster and more cost-effective
             # way of retrieving the existing record after a failed conditional write operation.
             record = exc.old_data_record or self._get_idempotency_record()
-            if is_replay and record is not None and record.status == "INPROGRESS":
+            if is_replay and record is not None and record.status == STATUS_CONSTANTS["INPROGRESS"]:
                 return self._get_function_response()
             # If a record is found, handle it for status
             if record:
@@ -296,7 +296,7 @@ def _get_function_response(self):
 
         else:
             try:
-                serialized_response: dict = self.output_serializer.to_dict(response) if response else None
+                serialized_response: dict = self.output_serializer.to_dict(response) if response is not None else None
                 self.persistence_store.save_success(data=self.data, result=serialized_response)
             except Exception as save_exception:
                 raise IdempotencyPersistenceLayerError(

aws_lambda_powertools/utilities/idempotency/persistence/redis.py

@@ -332,8 +332,8 @@ def _item_to_data_record(self, idempotency_key: str, item: dict[str, Any]) -> Da
             idempotency_key=idempotency_key,
             status=item[self.status_attr],
             in_progress_expiry_timestamp=in_progress_expiry_timestamp,
-            response_data=str(item.get(self.data_attr)),
-            payload_hash=str(item.get(self.validation_key_attr)),
+            response_data=item.get(self.data_attr, ""),
+            payload_hash=item.get(self.validation_key_attr, ""),
             expiry_timestamp=item.get("expiration"),
         )
 

tests/functional/idempotency/_redis/test_redis_layer.py

@@ -330,6 +330,25 @@ def test_item_to_datarecord_conversion(valid_record):
     assert record.in_progress_expiry_timestamp == item[layer.in_progress_expiry_attr]
 
 
+def test_item_to_datarecord_conversion_missing_optional_attributes(persistence_store_standalone_redis):
+    """
+    When data_attr or validation_key_attr is missing from Redis,
+    response_data and payload_hash should be empty string, not the string "None".
+    Regression test for: https://github.com/aws-powertools/powertools-lambda-python/issues/8090
+    """
+    idempotency_key = "test-func#abc123"
+    item = {
+        persistence_store_standalone_redis.status_attr: "COMPLETED",
+        persistence_store_standalone_redis.expiry_attr: 9999999999,
+        # data_attr and validation_key_attr intentionally absent
+    }
+
+    record = persistence_store_standalone_redis._item_to_data_record(idempotency_key, item)
+
+    assert record.response_data == ""
+    assert record.payload_hash == ""
+
+
 def test_idempotent_function_and_lambda_handler_redis_basic(
     persistence_store_standalone_redis: RedisCachePersistenceLayer,
     lambda_context,