# Feeds and indexes

> Register your product feed, create an index over it, and check that each scheduled ingest succeeded.

Most teams do this once, in the Console under **Developer tools → Settings → Feeds** and **Indexes**. The GraphQL mutations below do the same thing and need a team member's [session token](https://solenya.ai/docs/auth.md#sign-in-and-session-tokens): adding or deleting a feed needs `customer:write`, and creating an index needs `index:write:*`. A secret API key can't do either.

## Feed requirements

Solenya fetches your feed from its URL and checks every item. One invalid item fails the whole ingest, so check a new feed before adding it.

| Requirement | Detail |
|-------------|--------|
| Format | RSS 2.0 XML with the Google `g:` namespace, served plain or gzipped |
| Reachable | The URL is public and answers a `HEAD` request with a 2xx status. Keep it stable: Solenya re-fetches it on every ingest. |
| Fields Solenya uses | `id`, `title`, `description`, `link`, `image_link`, `price`, `availability`, `brand`; also `item_group_id`, `sale_price`, `product_type`, `size` and `color` when you have them |
| `price`, `sale_price` | An amount, a space and a currency code, such as `49.99 ZAR`. No thousands separators. |
| `id`, `item_group_id` | Letters, digits, spaces, `_`, `-` and `/` only |
| `availability` | `in_stock`, `out_of_stock`, `preorder` or `backorder`. `in stock` and `out of stock` are also accepted. |

See [Google's product data specification](https://support.google.com/merchants/answer/7052112) for what each field means.

## Feeds

### Add a feed

```graphql
mutation AddFeed {
  add_feed(feed_url: "https://shop.example.com/feed.xml") {
    url
  }
}
```

### List and delete feeds

```graphql
query Feeds {
  feeds {
    page {
      rows {
        record {
          url
        }
      }
    }
  }
}
```

```graphql
mutation DeleteFeed {
  delete_feed(feed_url: "https://shop.example.com/feed.xml") {
    url
  }
}
```

## Indexes

An index is a searchable copy of one feed with its own settings. Use several indexes over the same feed to keep staging apart from production, or to run different storefronts.

### Create an index

```graphql
mutation AddIndex {
  add_index(
    feed_list: ["https://shop.example.com/feed.xml"]
    name: "Production"
    ingest_schedule: "0 * * * *"
  ) {
    uuid
    name
  }
}
```

| Argument | Meaning |
|----------|---------|
| `feed_list` | The feed to index. Register it with `add_feed` first. |
| `name` | A label for you |
| `ingest_schedule` | How often Solenya re-fetches the feed, as a cron expression. Defaults to `"0 * * * *"`, every hour. |
| `tracking_mechanism` | Where shopper preferences come from: `inferred` (the default) learns from every item request; `event_based` learns from [`view_item` events](https://solenya.ai/docs/track.md#event-types). |
| `serving` | Optional [ranking settings](https://solenya.ai/docs/setup.md#serving-config). The two model ids can only be set later, with `update_index`. |

Save the returned `uuid`: it is the `Solenya-Index-UUID` header on every item query.

### Ingest schedule

A new index's schedule starts switched off, so no ingest runs until you switch it on. Do that by setting the schedule once with `update_index`:

```graphql
mutation StartIngest {
  update_index(index_uuid: "<index_uuid>", ingest_schedule: "0 * * * *") {
    uuid
  }
}
```

An invalid cron expression is rejected with `Invalid cron expression`.

### Check an ingest

Each index has a `deployment`: its schedule and its recent ingest runs.

```graphql
query IngestRuns {
  indexes {
    page {
      rows {
        record {
          uuid
          name
          deployment {
            schedules {
              active
              schedule {
                cron
                timezone
              }
            }
            runs(limit: 5) {
              state_type
              expected_start_time
              start_time
              end_time
              total_run_time
              logs(limit: 20) {
                timestamp
                level_name
                message
              }
            }
          }
        }
      }
    }
  }
}
```

| `state_type` | Meaning |
|--------------|---------|
| `SCHEDULED`, `PENDING` | Waiting to start |
| `RUNNING` | Fetching and indexing the feed |
| `COMPLETED` | Done; the catalogue is up to date |
| `FAILED`, `CRASHED` | The ingest did not finish. Read `logs` for the reason, such as an invalid item or an unreachable URL. |
| `CANCELLED`, `CANCELLING`, `PAUSED` | Stopped before it finished |

`runs` returns up to 10 runs unless you pass `limit`, and each run's `logs` up to 20 lines.

### Change an index

Pass only the fields you want to change. `feed_list` takes one feed per call, and a call that changes `feed_list` ignores `ingest_schedule`, so change the schedule in its own call.

```graphql
mutation ChangeFeed {
  update_index(
    index_uuid: "<index_uuid>"
    feed_list: ["https://shop.example.com/new-feed.xml"]
  ) {
    uuid
  }
}
```

```graphql
mutation DeleteIndex {
  delete_index(index_uuid: "<index_uuid>") {
    uuid
  }
}
```

### Serving config

An index's serving config is its set of ranking settings. Set them on the index with `update_index(serving: ...)`, or with `add_index` for every field except the two model ids. One request can override them with [`merchant_feed(serving: ...)`](https://solenya.ai/docs/search.md#changing-ranking-for-one-request), and inherits whatever it leaves out.

| Field | Affects | What it does |
|-------|---------|--------------|
| `query_model_id` | Text and image search | The model that turns a search into something comparable with your products. Must suit the index; see the models query below. |
| `score_cutoff` | Text search | Drops results whose relevance score is below this, from 0 to 1. `0` turns it off. It only works where Solenya has calibrated the index's scores, and not with a different `query_model_id`, so ask Solenya before relying on it. |
| `preference_model_id` | Personalisation | The model that learns each shopper's taste. Must suit the index. |
| `temperature` | Personalisation | How much variety personalised pages get. `0` or more; higher means more variety. |
| `default_prompt` | Category pages | The text that ranks a browse page with no search, product or image. |
| `cold_start` | Personalised category pages | What a shopper with no history sees: `default_prompt` (the default) ranks by the prompt above; `persona_mix` mixes in products from each of the preference model's personas. |

A request that sets a field its traffic can't use is rejected. To see which models an index can use, ask for `compatible_models` with `kind` set to `embedding` (search) or `preference` (personalisation):

```graphql
query CompatibleModels {
  indexes {
    page {
      rows {
        record {
          uuid
          serving {
            query_model_id
            preference_model_id
            temperature
          }
          compatible_models(kind: preference) {
            id
            repo_id
            embedding_dimension
          }
        }
      }
    }
  }
}
```

### List indexes

```graphql
query Indexes {
  indexes {
    page {
      rows {
        record {
          uuid
          name
          tracking_mechanism
          feeds {
            url
          }
        }
      }
    }
  }
}
```

`ingest_schedule` on an index is deprecated. Read the schedule from `deployment.schedules` instead.
