Cohere Release Notes

Aya Expanse is a multilingual large language model that is designed to expand the number of languages covered by generative AI. It is optimized to perform well in 23 languages, including Arabic, Chinese (simplified & traditional), Czech, Dutch, English, French, German, Greek, Hebrew, Russian, Spanish, and more.

Now, you can talk to Aya Expanse directly in the popular messaging service WhatsApp! All of Aya's functionality is avaible through the app, and you can find more details in the Aya Family of Models section.

We're thrilled to announce the release of Command R7B, the smallest, fastest, and final model in our R family of enterprise-focused large language models (LLMs). With a context window of 128K, Command R7B offers state-of-the-art performance across a variety of real-world tasks, and is designed for use cases in which speed, cost, and compute are important. Specifically, Command R7B is excellent for retrieval-augmented generation, tool use, and agentic applications where complex reasoning, multiple actions, and information-seeking are important for success.

Command R7B is available today on the Cohere Platform as well as accessible on HuggingFace, or you can access it in the SDK with command-r7b-12-2024. For more information, check out our dedicated blog post.

We're pleased to announce the release of Rerank 3.5 our newest and most performant foundational model for ranking. Rerank 3.5 has a context length of 4096, SOTA performance on Multilingual Retrieval tasks and Reasoning Capabilities. In addition, Rerank 3.5 has SOTA performance on BEIR and domains such as Finance, E-commerce, Hospitality, Project Management, and Email/Messaging Retrieval tasks.

In the rest of these release notes, we’ll provide more details about changes to the api.

Technical Details

API Changes

Along with the model, we are releasing V2 of the Rerank API. It includes the following major changes:

• model is now a required parameter
• max_chunks_per_doc has been replaced by max_tokens_per_doc; max_tokens_per_doc will determine the maximum amount of tokens a document can have before truncation. The default value for max_tokens_per_doc is 4096.
• support for passing a list of objects for the documents parameter has been removed - if your documents contain structured data, for best performance we recommend formatting them as YAML strings.

Example request

POST https://api.cohere.ai/v2/rerank
{
    "model": "rerank-v3.5",
    "query": "What is the capital of the United States?",
    "top_n": 3,
    "documents": ["Carson City is the capital city of the American state of Nevada.", …

Today, we're pleased to announce that we have added Structured Outputs support for tool use in the Chat API.

In addition to supporting Structured Outputs with JSON generation via the response_format parameter, Structured Outputs will be available with Tools as well via the strict_tools parameter.

Setting strict_tools to true ensures that tool calls will follow the provided tool schema exactly. This means the tool calls are guaranteed to adhere to the tool names, parameter names, parameter data types, and required parameters, without the risk of hallucinations.

See the Structured Outputs documentation to learn more.

Today we’re announcing updates to our embed-v3.0 family of models. These models now have the ability to process images into embeddings. There is no change to existing text capabilities which means there is no need to re-embed texts you have already processed with our embed-v3.0 models.

In the rest of these release notes, we’ll provide more details about technical enhancements, new features, and new pricing.

Technical Details

API Changes

The Embed API has two major changes:

• Introduced a new input_type called image
• Introduced a new parameter called images

Example request on how to process

POST https://api.cohere.ai/v1/embed
{
    "model": "embed-multilingual-v3.0",
    "input_type": "image",
    "embedding_types": ["float"],
    "images": [enc_img]
}

Restrictions

• The API only accepts images in the base format of the following: png, jpeg,Webp, and gif
• Image embeddings currently does not support batching so the max images sent per request is 1
• The maximum image sizez is 5mb
• The images parameter only accepts a base64 encoded image formatted as a Data Url

We're excited to introduce improvements to our Chat, Classify, Embed, and Rerank APIs in a major version upgrade, making it easier and faster to build with Cohere. We are also releasing new versions of our Python, TypeScript, Java, and Go SDKs which feature cohere.ClientV2 for access to the new API.

New at a glance

• V2 Chat, Classify, Embed, and Rerank: model is a required parameter
• V2 Embed: embedding_types is a required parameter
• V2 Chat: Message and chat history are combined in a single messages array
• V2 Chat: Tools are defined in JSON schema
• V2 Chat: Introduces tool_call_ids to match tool calls with tool results
• V2 Chat: documentssupports a list of strings or a list of objects with document metadata
• V2 Chat streaming: Uses server-sent events

Other updates

We are simplifying the Chat API by removing support for the following parameters available in V1:

• search_queries_only, which generates only a search query given a user’s message input. search_queries_only is not supported in the V2 Chat API today, but will be supported at a later date. …

Today we’re announcing updates to our flagship generative AI model series: Command R and Command R+. These models demonstrate improved performance on a variety of tasks.

The latest model versions are designated with timestamps, as follows:

• The updated Command R is command-r-08-2024 on the API.
• The updated Command R+ is command-r-plus-08-2024 on the API.

In the rest of these release notes, we’ll provide more details about technical enhancements, new features, and new pricing.

Technical Details

command-r-08-2024 shows improved performance for multilingual retrieval-augmented generation (RAG) and tool use. More broadly, command-r-08-2024 is better at math, code and reasoning and is competitive with the previous version of the larger Command R+ model.

command-r-08-2024 delivers around 50% higher throughput and 20% lower latencies as compared to the previous Command R version, while cutting the hardware footprint required to serve the model by half. Similarly, command-r-plus-08-2024 delivers roughly 50% higher throughput and 25% lower latencies as compared to the previous Command R+ version, while keeping the hardware footprint the same.

Both models include the following feature improvements:

• For tool use, command-r-08-2024 and command-r-plus-08-2024 have demonstrated improved decision-making around which tool to use in which context, and whether or not to use a tool.
• Improved instruction following in the preamble. …

Users can now force command-nightlyto generate outputs in JSON objects by setting the response_format parameter in the Chat API. Users can also specify a JSON schema for the output.

This feature is available across all of Cohere's SDKs (Python, Typescript, Java, Go).

Example request for forcing JSON response format:

POST https://api.cohere.ai/v1/chat
{
    "message": "Generate a JSON that represents a person, with name and age",
    "model": "command-nightly",
    "response_format": {
        "type": "json_object"
    }
}

Example request for forcing JSON response format in user defined schema:

POST https://api.cohere.ai/v1/chat
{
    "message": "Generate a JSON that represents a person, with name and age",
    "model": "command-nightly",
    "response_format": {
        "type": "json_object",
        "schema": {
            "type": "object",
            "required": ["name", "age"],
            "properties": {
                "name": { "type": "string" },
                "age": { "type": "integer" }
            }
        }
    }
}

Currently only compatible with `command-nightly model.

Multi-step tool use now default in Chat API

Tool use is a technique which allows developers to connect Cohere's Command family of models to external tools like search engines, APIs, functions, databases, etc. It comes in two variants, single-step and multi-step, both of which are available through Cohere's Chat API.

As of today, tool use will now be multi-step by default. Here are some resources to help you get started:

• Check out our multi-step tool use guide.
• Experiment with multi-step tool use with this notebook.

We've published additional docs

Cohere's models and functionality are always improving, and we've recently dropped the following guides to help you make full use of our offering:

• Predictable outputs - Information about the seed parameter has been added, giving you more control over the predictability of the text generated by Cohere models. …

Today, we are introducing Command R, a new LLM aimed at large-scale production workloads. Command R targets the emerging “scalable” category of models that balance high efficiency with strong accuracy, enabling companies to move beyond proof of concept, and into production.

Command R is a generative model optimized for long context tasks such as retrieval-augmented generation (RAG) and using external APIs and tools. It is designed to work in concert with our industry-leading Embed and Rerank models to provide best-in-class integration for RAG applications and excel at enterprise use cases. As a model built for companies to implement at scale, Command R boasts:

• Strong accuracy on RAG and Tool Use
• Low latency, and high throughput
• Longer 128k context and lower pricing
• Strong capabilities across 10 key languages
• Model weights available on HuggingFace for research and evaluation

For more information, check out the official blog post or the Command R documentation.

Apply Cohere's AI with Connectors

One of the most exciting applications of generative AI is known as "retrieval augmented generation" (RAG). This refers to the practice of grounding the outputs of a large language model (LLM) by offering it resources -- like your internal technical documentation, chat logs, etc. -- from which to draw as it formulates its replies.

Cohere has made it much easier to utilize RAG in bespoke applications via Connectors. As the name implies, Connectors allow you to connect Cohere's generative AI platform up to whatever resources you'd like it to ground on, facilitating the creation of a wide variety of applications -- customer service chatbots, internal tutors, or whatever else you want to build.

Our docs cover how to create and deploy connectors, how to manage your connectors, how to handle authentication, and more!

Expanded Fine-tuning Functionality …

We're Releasing co.chat() and the Chat + RAG Playground

We're pleased to announce that we've released our co.chat() beta! Of particular importance is the fact that the co.chat() API is able to utilize retrieval augmented generation (RAG), meaning developers can provide sources of context that inform and ground the model's output.

This represents a leap forward in the accuracy, verifiability, and timeliness of our generative AI offering. For our public beta, developers can connect co.chat() to web search or plain text documents.

Access to the co.chat() public beta is available through an API key included with a Cohere account.

Our Command Model has Been Updated

We've updated both the command and command-light models. Expect improved question answering, generation quality, rewriting and conversational capabilities.

New Rate Limits

For all trial keys and all endpoints, there is now a rate limit of 5000 calls per month.