Fixtures#
Streaming fixture loaders, batch iterators, sharded writers, and format aliases.
Fixture Formats & Readers#
- advanced_alchemy.utils.fixtures.FixtureFormat#
Supported fixture file serialization format (
"json","jsonl", or"csv").alias of
Literal[‘json’, ‘jsonl’, ‘csv’]
- advanced_alchemy.utils.fixtures.FixtureCompression#
Supported fixture archive compression format (
"gzip","zip", orNone).alias of
Literal[‘gzip’, ‘zip’]
- advanced_alchemy.utils.fixtures.open_fixture_sync(fixtures_path, fixture_name)[source]#
Load a JSON, JSONL, or CSV fixture synchronously.
Supports plain files, gzip-compressed files (
.gz), and zip archives (.zip). Formats are resolved in priority order:json>jsonl>csv. Both lowercase and uppercase fixture names are checked according to format and compression rules. Text is decoded usingutf-8-sig.Unsharded
jsonfixtures return the decoded top-level JSON value unchanged. Unshardedjsonlandcsvfixtures return a list of decoded records or dictionaries.- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Directory path containing fixture files. Can be a
pathlib.Pathoranyio.Pathinstance.fixture_name (str) – Fixture base name without file extension.
- Raises:
FileNotFoundError – If no matching fixture file exists.
OSError – If an I/O or decompression error occurs while reading the file.
ValueError – If a zip archive contains no matching member, a JSONL line contains invalid JSON, or a JSON fixture read as records is not an array.
- Returns:
Decoded JSON data or a list of records.
- Return type:
Any
Examples
>>> from pathlib import Path >>> fixtures_path = Path("./fixtures") >>> data = open_fixture_sync(fixtures_path, "users") >>> print(data) [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]
- async advanced_alchemy.utils.fixtures.open_fixture_async(fixtures_path, fixture_name)[source]#
Load a JSON, JSONL, or CSV fixture asynchronously.
Offloads synchronous fixture resolution, decompression, and parsing to a worker thread via
async_().- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Directory path containing fixture files. Can be a
pathlib.Pathoranyio.Pathinstance.fixture_name (str) – Fixture base name without file extension.
- Raises:
FileNotFoundError – If no matching fixture file exists.
OSError – If an I/O or decompression error occurs while reading the file.
ValueError – If a zip archive contains no matching member, a JSONL line contains invalid JSON, or a JSON fixture read as records is not an array.
- Returns:
Decoded JSON data or a list of records.
- Return type:
Any
Examples
>>> from pathlib import Path >>> fixtures_path = Path("./fixtures") >>> data = await open_fixture_async(fixtures_path, "users") >>> print(data) [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]
Streaming, Writing & Legacy Alias#
- advanced_alchemy.utils.fixtures.iter_fixture_sync(fixtures_path, fixture_name, *, batch_size=1000)[source]#
Stream decoded fixture records in non-empty batches synchronously.
Validates
batch_sizeand resolves the fixture file or numbered shards eagerly at call time, then yields non-empty batches of at mostbatch_sizerecords across shard boundaries. Closing the returned iterator early releases any open file handle immediately.- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Directory path containing fixture files. Can be a
pathlib.Pathoranyio.Pathinstance.fixture_name (str) – Fixture base name without file extension.
batch_size (int) – Maximum number of records per yielded batch. Must be at least 1.
- Raises:
FileNotFoundError – If no matching fixture file exists.
OSError – If an I/O or decompression error occurs while reading a file.
ValueError – If
batch_sizeis less than 1, both an unsharded file and numbered shards exist, shard validation fails, a zip archive contains no matching member, a JSONL line contains invalid JSON, or a JSON fixture is not an array.
- Returns:
Iterator yielding non-empty lists of records.
- Return type:
Iterator[list[Any]]
- async advanced_alchemy.utils.fixtures.iter_fixture_async(fixtures_path, fixture_name, *, batch_size=1000)[source]#
Stream decoded fixture records in non-empty batches asynchronously.
Offloads fixture resolution, batch reads, and generator teardown to worker threads via
async_(). Validation and resolution run on the first iteration step. Callers exiting early must callaclose()or consume the iterator completely so underlying file handles are closed promptly.- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Directory path containing fixture files. Can be a
pathlib.Pathoranyio.Pathinstance.fixture_name (str) – Fixture base name without file extension.
batch_size (int) – Maximum number of records per yielded batch. Must be at least 1.
- Raises:
FileNotFoundError – If no matching fixture file exists.
OSError – If an I/O or decompression error occurs while reading a file.
ValueError – If
batch_sizeis less than 1, both an unsharded file and numbered shards exist, shard validation fails, a zip archive contains no matching member, a JSONL line contains invalid JSON, or a JSON fixture is not an array.
- Yields:
list[Any]– Non-empty batch of decoded records.- Return type:
AsyncIterator[list[Any]]
- advanced_alchemy.utils.fixtures.write_fixture_sync(
- fixtures_path,
- fixture_name,
- records,
- *,
- file_format='json',
- compression=None,
- max_bytes=None,
- max_records=None,
Write fixture records synchronously in JSON, JSONL, or CSV format.
Streams encoded records into temporary
.tmpfiles, atomically renames temporary files into place upon completion, and removes any stale fixture files matchingfixture_nameacross all formats, compressions, and shard layouts.- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Target directory path for fixture files. Created if it does not exist. Can be a
pathlib.Pathoranyio.Path.fixture_name (str) – Fixture base name without file extension.
records (Iterable[Any]) – Iterable of records to write.
file_format ('json' | 'jsonl' | 'csv') – Output format (
"json","jsonl", or"csv").compression (Optional[FixtureCompression]) – Optional compression (
"gzip","zip", orNone).max_bytes (Optional[int]) – Optional maximum on-disk byte size per shard. Must be at least 1.
max_records (Optional[int]) – Optional maximum record count per shard. Must be at least 1.
- Raises:
ValueError – If
file_format,compression,max_bytes, ormax_recordsis invalid, or if CSV records introduce keys absent from the header.TypeError – If
file_formatis"csv"and a record is not a mapping.OSError – If writing, deleting stale files, or renaming files fails.
- Returns:
Final written fixture paths in shard order.
- Return type:
list[Path]
Examples
>>> from pathlib import Path >>> fixtures_path = Path("./fixtures") >>> paths = write_fixture_sync( ... fixtures_path, ... "users", ... [{"id": 1, "name": "Alice"}], ... file_format="jsonl", ... compression="gzip", ... )
- async advanced_alchemy.utils.fixtures.write_fixture_async(
- fixtures_path,
- fixture_name,
- records,
- *,
- file_format='json',
- compression=None,
- max_bytes=None,
- max_records=None,
- batch_size=1000,
Write fixture records asynchronously in JSON, JSONL, or CSV format.
Streams records from a synchronous or asynchronous iterable in batches of
batch_sizeinto temporary.tmpfiles, offloads writer construction, batch writes, commit, and abort to worker threads viaasync_(), atomically renames temporary files into place on completion, and removes stale fixture files matchingfixture_name.- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Target directory path for fixture files. Created if it does not exist. Can be a
pathlib.Pathoranyio.Path.fixture_name (str) – Fixture base name without file extension.
records (Union[Iterable[Any], AsyncIterable[Any]]) – Synchronous or asynchronous iterable of records to write.
file_format ('json' | 'jsonl' | 'csv') – Output format (
"json","jsonl", or"csv").compression (Optional[FixtureCompression]) – Optional compression (
"gzip","zip", orNone).max_bytes (Optional[int]) – Optional maximum on-disk byte size per shard. Must be at least 1.
max_records (Optional[int]) – Optional maximum record count per shard. Must be at least 1.
batch_size (int) – Maximum number of records buffered before offloading a batch write to a worker thread. Must be at least 1.
- Raises:
ValueError – If
batch_size,file_format,compression,max_bytes, ormax_recordsis invalid, or if CSV records introduce keys absent from the header.TypeError – If
file_formatis"csv"and a record is not a mapping.OSError – If writing, deleting stale files, or renaming files fails.
- Returns:
Final written fixture paths in shard order.
- Return type:
list[Path]
Examples
>>> from pathlib import Path >>> async def stream_users(): ... yield {"id": 1, "name": "Alice"} >>> paths = await write_fixture_async( ... Path("./fixtures"), ... "users", ... stream_users(), ... file_format="jsonl", ... compression="gzip", ... )
- advanced_alchemy.utils.fixtures.open_fixture(fixtures_path, fixture_name)[source]#
Load a JSON, JSONL, or CSV fixture synchronously.
Deprecated since 1.13.0 and scheduled for removal in 2.0. Use
open_fixture_sync()instead.- Parameters:
fixtures_path (Union[Path, AsyncPath]) – Directory path containing fixture files. Can be a
pathlib.Pathoranyio.Pathinstance.fixture_name (str) – Fixture base name without file extension.
- Raises:
FileNotFoundError – If no matching fixture file exists.
OSError – If an I/O or decompression error occurs while reading the file.
ValueError – If a zip archive contains no matching member, a JSONL line contains invalid JSON, or a JSON fixture read as records is not an array.
- Returns:
Decoded JSON data or a list of records.
- Return type:
Any