Problem Statement
A regression introduced in version 3.1.0 caused Index.where to raise an AssertionError when an Index object was passed as the other argument. The issue arose because the internal engine incorrectly assumed label alignment when the other argument was an Index, causing a failure, especially when Extension Array dtypes (like Int64) were involved.
The Technical Fix
The fix was implemented in pandas/core/generic.py and involved updates to both the internal _where logic and the public where docstring.
Inside _where, I updated the fallback unwrapping block (which catches standard Index objects) by adding extract_range=True to the extract_array call. This ensures that index types—particularly RangeIndex—are properly extracted and unwrapped before being passed to the internal engine, bypassing the AssertionError.
elif not isinstance(other, (MultiIndex, NDFrame)):
# mainly just catching Index here
other = extract_array(other, extract_numpy=True, extract_range=True)
Based on reviewer feedback from jbrockmendel and rhshadrach, extract_array was used instead of to_numpy(). This is critical for safely preserving Extension Array metadata and preventing unintended downcasting to standard NumPy arrays.
I also updated the docstring for the where method to explicitly declare array-like and Index as valid types for the other parameter, improving the API documentation.
Testing & Validation
To ensure robustness and prevent future regressions, I added two new unit tests in pandas/tests/indexes/base_class/test_where.py:
test_where_index_with_index: Verifies thatIndex.whereexecutes successfully whenotheris aRangeIndex(Index(range(48))).test_where_index_with_ea_index: Verifies thatIndex.wherecorrectly preserves Extension Array types (specificallydtype="Int64") when an EA Index is passed asother, ensuring no unintended downcasting occurs.
Tracing an AssertionError through complex internal dispatch mechanisms in the pandas core.
Distinguishing between safe type-preserving utilities (extract_array) and standard conversion methods (to_numpy).
Learned that safe handling of Extension Arrays requires specialized utilities rather than generic NumPy conversions.
Gained insight into how Index.where handles label alignment versus positional replacement.