Organize documents

Group documents into collections and tag them with metadata. A search, or an agent, can then look at just the documents you mean: see Choose the documents.

Collections

A collection is a named group of documents, as a folder holds files: a project, a customer, a knowledge base, which you search or give an agent whole.

A document is in at most one collection. Adding one to a collection moves it there from any other.
The same file uploaded into another collection is another document there, with its own metadata; uploaded again into the same one, it is the document already there.
Deleting a collection deletes its documents. To keep some, move them elsewhere first.
Metadata describes each document: what it is, to filter by. A collection says which set it belongs to.

reports = client.collections.create("Annual reports")

doc = client.documents.upload("3M_2018_10K.pdf", collection=reports)  # a new document, in it
client.collections.add(reports, ["doc_60a07bcef3094b68806070f2"])     # or move documents you have into it

for doc in client.documents.list(collection=reports):
    print(doc.id, doc.filename)

Metadata

Give each document your own keys, on upload or later: up to 16, each a string, number or boolean. Setting a key to null removes it.

doc = client.documents.upload("3M_2018_10K.pdf", metadata={"ticker": "MMM", "year": 2018})
client.documents.update(doc, metadata={"audited": True})  # None removes a key

recent = client.documents.list(filter={"year": {"$gte": 2020}})

To see which keys your documents have, with their types and values:

schema = client.collections.schema(reports)  # or client.documents.schema(filter=...)
for key in schema.keys:
    print(key.key, key.types, key.documents, key.values or (key.min, key.max))

Filters

A filter picks documents by metadata; a document must meet every condition.

Condition	Holds when the key's value	Example
`$eq`	equals it; a value alone means the same	`{"ticker": "MMM"}`
`$ne`	does not equal it, or the key is missing	`{"ticker": {"$ne": "MMM"}}`
`$in`	is one of them (1 to 100 values)	`{"ticker": {"$in": ["MMM", "AAPL"]}}`
`$nin`	is none of them, or the key is missing	`{"ticker": {"$nin": ["MMM"]}}`
`$gt, $gte`	is a number greater than (or equal to) it	`{"year": {"$gte": 2020}}`
`$lt, $lte`	is a number less than (or equal to) it	`{"year": {"$lt": 2025}}`
`$and`	meets every filter in the list	`{"$and": [{"year": 2018}, {"audited": true}]}`
`$or`	meets any filter in the list	`{"$or": [{"ticker": "MMM"}, {"region": "US"}]}`

Values compare exactly: "2018" is not 2018.
$gt, $gte, $lt and $lte compare numbers; store dates as numbers, such as 20240301.
A document without the key meets only $ne and $nin.
Limits and every field: API reference.

An agent sees your metadata's keys and values, and writes filters itself: with metadata, it finds the documents a question names for free. See Agent tools.