# Changelog

## Score

- CAI 63 → 55 (-8.7)
- Rubric changed (rubric-2026.10.1 → rubric-2026.10.5) — scores are not directly comparable.

## Lenses

- Code Health 74 → 70 (-3.4)
- Architecture 89 → 89 (+0.0)
- Maturity 57 → 35 (-22.4)
- Readiness 72 → 72 (+0.0)
- Security 60 → 68 (+8.0)
- Performance 89 → 89 (+0.0)

## Resolved (11)

- Base-context workflow trigger runs with an unscoped token
- Duplicated block (10 lines × 2) (src/datasets/arrow_dataset.py)
- Duplicated block (12 lines × 2) (src/datasets/features/image.py)
- Duplicated block (29–30 lines × 2) (src/datasets/arrow_dataset.py)
- Duplicated block (29–30 lines × 2) (src/datasets/arrow_dataset.py)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/datasets/arrow_dataset.py)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/datasets/formatting/jax_formatter.py)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/datasets/io/csv.py)
- Off-boarding risk: anonymized user #1
- PR-triggered workflow without a permissions block
- Workflow holding a long-lived secret is unscoped

## New (30)

- Base-context workflow trigger runs with an unscoped token (.github/workflows/upload_pr_documentation.yml)
- Dependency hygiene PARTLY measured — Python dependencies read, no exact pin to grade for currency
- Duplicated block (12 lines × 2) (src/datasets/arrow_dataset.py)
- Duplicated block (13 lines × 2) (src/datasets/arrow_dataset.py)
- Duplicated block (13 lines × 3) (src/datasets/features/image.py)
- Duplicated block (29 lines × 2) (src/datasets/arrow_dataset.py)
- IterableDataset._write_parquet_shards (cognitive 34) (src/datasets/iterable_dataset.py)
- IterableDataset._write_parquet_shards (cyclomatic 16) (src/datasets/iterable_dataset.py)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/datasets/arrow_dataset.py)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/datasets/formatting/jax_formatter.py)
- Members sharing a duplicated core (4 members, 50+ identical tokens) (src/datasets/io/csv.py)
- Off-boarding risk: anonymized user #1
- Orphaned file (benchmarks/benchmark_array_xd.py)
- Orphaned file (benchmarks/benchmark_iterating.py)
- Orphaned file (benchmarks/benchmark_map_filter.py)
- Orphaned file (src/datasets/formatting/polars_formatter.py)
- Orphaned file (src/datasets/inspect.py)
- Orphaned file (src/datasets/naming.py)
- Orphaned file (src/datasets/packaged_modules/iceberg/iceberg.py)
- Orphaned file (src/datasets/packaged_modules/spark/spark.py)
- …and 10 more

## Changes since last survey

- 32 commits — 21 feature/other, 11 fixes

## By area

- src/datasets — 24 commits
- (root) — 5 commits
- docs/source — 2 commits
- .github/workflows — 1 commit

## Notable commits

- fix: Fix IterableDataset.filter with arrow, pandas and polars formats (#8725)
- fix: Fix Json.cast_storage double-encoding large_string and string_view columns (#8682)
- fix: Fix audio array handling in preprocess_function (#8731)
- fix: Fix epoch iteration after skipping or taking a streaming subset (#8707)
- fix: Fix indices buffer loading in Dataset.from_buffer (#8840)
- fix: Fix inverted span for minus-strand multi-exon GenBank locations (#8677)
- fix: Fix push_to_hub dropping rows when num_proc exceeds num_shards (#8795)
- fix: Fix repeated Arrow rebatch checkpoint restoration (#8727)
- fix: fix default iterable dataset dict and polars exports (#8632)
- fix: fix(json): defensively recover invalid Arrow tables in JSON loader (#5531) (#8521)
- fix: harbor: fix paths on windows (#8739)
- change: Allow IterableDataset.push_to_hub num_shards above dataset.num_shards (#8716)
- change: Batch in arrow when the source is arrow-backed, and let `_consolidate` type numpy scalars (#8665)
- change: Bump doc-builder workflow pin (#8753)
- change: Bump huggingface_hub upper bound to <3.0 (datasets) (#8678)
- change: Close the xopen handle in xpandas_read_csv (#8786)
- change: Decode remote NIfTI files through the bytes path (#8737)
- change: Don't import aiohttp when importing datasets (#8680)
- change: Fast streaming decode for image/audio/video/etc. folders datasets (#8791)
- change: Implement __arrow_array__ on PandasArrayExtensionArray so ArrayXD columns can go back from pandas to Arrow (#8701)
- …and 12 more