Integrate Custom Data
Make data produced in your application available through SID
Overview
While using your application, your users constantly generate content through their interactions, which can provide valuable context in the future.
For example, let’s imagine that we want to build an AI chat application that can remember past conversations. There are two main aspects to this:
- Ingesting past conversations
- Retrieving relevant conversation data
The custom data insertion functionality can be used to achieve exactly this: Whenever the user finishes a conversation, it is converted into text and sent to SID, where it is processed, chunked and indexed. It can then be retrieved through the query functionality.
Inserting Data
You can ingest your custom data by POST
ing JSON data to the
ingest endpoint
as shown below:
{
"data": [
{
"name": "Example Item 1",
"text": "This is the raw text for Example Item 1"
},
{
"name": "Example Item 2",
"text": "This is the raw text for Example Item 2"
},
{
"name": "Long Item",
"text": "Lorem ipsum dolor sit amet, ..."
}
]
}
The text
field can be arbitrarily long. If necessary, it will be partitioned during indexing.
Retrieving Data
After you ingest your custom data, it will become available for retrieval through the standard query endpoint.
Items ingested in this manner will have a kind
value of custom
.
Deleting Data
For a particular user, you can delete all custom data ingested through your service by making an empty post request to the clear endpoint.
It is currently not possible to delete individual items. If this is a feature you would like to see, please reach out and let us know!
Was this page helpful?