Skip to content

Numerical Python·Lesson 21 of 67·intermediate

Indexing, slicing and boolean masks

Select along several axes at once, filter with boolean masks instead of loops, and find the top-k results the way every retrieval system does.


In this lesson · 18 min

  • →Index and slice arrays along more than one axis
  • →Select arbitrary rows with integer-array indexing
  • →Filter with boolean masks, and combine them correctly
  • →Use where, argmax, argsort and argpartition
  • →Assign through a mask to change only the elements you selected

Selecting the right elements is most of what you do with an array. Doing it with indexing rather than a loop is both faster and clearer — and it is exactly how a retrieval system finds its top results.

Along one axis

PythonPython not started

Along several axes

A comma separates the axes. arr[row, column] is one element; a slice in either position keeps that axis:

PythonPython not started

... stands for “all the remaining axes”, which keeps high-dimensional indexing readable:

PythonPython not started

Integer-array indexing

Passing a list or array of indices picks exactly those elements, in that order — and repeats are allowed:

PythonPython not started

Unlike a slice, this copies. That is usually what you want when you are gathering the documents a search returned.

Boolean masks

Comparing an array gives an array of booleans, and indexing with it keeps the True positions:

PythonPython not started

Combine masks with & (and), | (or) and ~ (not) — each condition in its own parentheses:

PythonPython not started
PythonPython not started

Filtering rows of a batch

A mask over one axis selects along that axis, which is how you keep the rows that matter:

PythonPython not started

where

np.where(condition, if_true, if_false) chooses elementwise, with no loop and no mask juggling:

PythonPython not started

Finding positions: argmax, argsort, argpartition

The arg family returns indices rather than values, which is what you need when the array you are searching is parallel to another one you want to look things up in:

PythonPython not started

Top-k retrieval is exactly this, and argpartition does it without sorting everything — which matters when you are ranking a million candidates and want three:

PythonPython not started

Assigning through a selection

Indexing works on the left of an = too, and only the selected elements change:

PythonPython not started

Try it yourself

Exercise 1

Set top_docs to the three highest-scoring document names, best first, and top_scores to their scores.

Your turnPython not started
Exercise 2

Keep only the rows of embeddings whose quality is at least 0.7 and whose length is under 2000. Put them in kept, and their ids in kept_ids.

Your turnPython not started

Check yourself

Why does (scores > 0.6) and (lengths > 100) raise ValueError, while & works?

Check yourself

You need the 10 best of a million similarity scores. Why argpartition rather than argsort?

Recap

  • Separate axes with a comma: arr[row, column], and ... fills in the axes you did not name.
  • Integer-array indexing picks arbitrary elements in any order, and copies.
  • A comparison gives a boolean mask; index with it to filter. Combine masks with &, | and ~, parenthesising each comparison — never and/or.
  • np.where chooses elementwise; with one argument it returns the matching indices.
  • argmax, argsort and argpartition return positions, so you can look them up in a parallel array — the last step of every vector search.
  • Selections work on the left of =, changing only what they selected.