> ## Documentation Index
> Fetch the complete documentation index at: https://www.integrate.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# ETL: Python Transformation

> Use the Python Transformation to run custom Python code for data processing as part of your Integrate.io ETL pipeline. Supports pandas.

The Python Transformation component allows you to run custom Python code as part of your Integrate.io ETL pipeline. It transforms records without changing the schema of the input data.

## Overview

* Accepts an array of JSON records as input
* Requires a defined `transform` function
* Input and output schemas must match exactly
* Useful for string manipulation, cleaning, enrichment, or any custom logic
* You can remove records from the array (filtering), but returned records must follow the original schema
* You may return an empty array, but not a different schema

## Example Transformation Code

```python theme={null}
import json

def transform(event, context):
    """
    Transforms input records by converting all string values to uppercase.
    The output schema must match the input schema exactly.
    """
    try:
        data = event
        results = []

        for record in data:
            transformed_record = {}
            for key, value in record.items():
                if isinstance(value, str):
                    transformed_record[key] = value.upper()
                else:
                    transformed_record[key] = value
            results.append(transformed_record)

        return results

    except Exception as e:
        raise e
```

## Example Input and Output

**Input:**

```json theme={null}
[
  { "id": 1, "name": "Belgian Waffles", "price": 5.95 },
  { "id": 2, "name": "Pancakes", "price": 4.95 }
]
```

**Output:**

```json theme={null}
[
  { "id": 1, "name": "BELGIAN WAFFLES", "price": 5.95 },
  { "id": 2, "name": "PANCAKES", "price": 4.95 }
]
```

## Configuration

**Batch Size:** Define the number of records per batch. The total payload size should not exceed 6 MB.

## Testing

Test your transformation in the Package Designer: provide a sample payload in table format, click **Run Code**, and view the transformed output. The test table mirrors the schema of the connected input component.

## Variables

Use package, secret, and global variables with Python import format:

```python theme={null}
from package_variables import variable_name
```

All variables (including secrets and globals) are evaluated before passing to the function.

## Best Practices

* Always ensure output matches input schema (field names and types).
* Valid to return fewer records or an empty array for filtering.
* Catch and log exceptions to help debug errors.
* Break complex logic into helper functions within the same script.