# Build with XMTP > XMTP is a secure and decentralized protocol for building communication platforms for the next phase of the internet. ## Privacy policy `docs.xmtp.org` does not use cookies and does not collect any personal data or personally-identifiable information (PII). The site does use Plausible analytics to track overall trends in website traffic. Plausible does not track individual visitors. All analytics data is aggregated data only and it contains no personal information. The site uses Plausible analytics for the sole purpose of helping site contributors understand how to best deliver content that serves the needs of the XMTP community. To learn more about the data Plausible tracks, see the [Plausible Data Policy](https://plausible.io/data-policy). If you have any questions about this privacy policy, post to the [XMTP Technical Forums](https://community.xmtp.org/). ## Terms of service Our content licensing policies are based on those of the [Google Developer](https://developers.google.com/site-policies) website. We are pleased to license much of the documentation on `docs.xmtp.org` under terms that explicitly encourage people to take, modify, reuse, re-purpose, and remix this content as they see fit. You will find the following notice at the bottom of pages on `docs.xmtp.org`: > CC BY 4.0 This means that except as otherwise noted, page content is licensed under the [Creative Commons Attribution 4.0 License](https://creativecommons.org/licenses/by/4.0/) and code samples are licensed under the [MIT License](http://opensource.org/licenses/MIT). When you see a page with this notice, you are free to use [nearly everything](#what-is-not-licensed) on the page in your own creations. For example, you could quote the text in a book, cut-and-paste sections to your blog, record it as an audiobook for the visually impaired, or even translate it into Swahili. Really. That's what open content licenses are all about. We just ask that you give us [attribution](#attribution) when you reuse our work. If you have any questions about these terms of service, post to the [XMTP Technical Forums](https://community.xmtp.org/). ### What is *not* licensed? We say “nearly everything” as there are a few simple conditions that apply. Trademarks and other brand features are not included in this license. In some cases, a page might include content consisting of images, audio or video material, or a link to content on a different webpage (such as videos or slide decks). This content is not covered by the license, unless specifically noted. ### Attribution Proper attribution is required when you reuse or create modified versions of content that appears on a page made available under the terms of the Creative Commons Attribution license. The complete requirements for attribution can be found in section 3 of the [Creative Commons legal code](https://creativecommons.org/licenses/by/4.0/legalcode). In practice, we ask that you provide attribution to XMTP to the best of the ability of the medium in which you are producing the work. There are several typical ways in which this might apply: #### Exact reproductions If your online work *exactly reproduces* text or images from this site, in whole or in part, please include a paragraph at the bottom of your page that reads: > Portions of this page are reproduced from work created and shared by XMTP and used according to terms described in the [Creative Commons 4.0 Attribution License](https://creativecommons.org/licenses/by/4.0/). Also, please link back to the original source page so that readers can refer to it for more information. #### Modified versions If your online work shows *modified* text or images based on the content from this site, please include a paragraph at the bottom of your page that reads: > Portions of this page are modifications based on work created and shared by XMTP and used according to terms described in the [Creative Commons 4.0 Attribution License](https://creativecommons.org/licenses/by/4.0/). Again, please link back to the original source page so that readers can refer to it for more information. This is even more important when the content has been modified. #### Other media If you produce non-hypertext works, such as books, audio, or video, we ask that you make a best effort to include a spoken or written attribution in the spirit of the messages above. ## Page not found This page may have been moved, renamed, or removed. Our docs are actively maintained, so URLs occasionally change. Please use the search field or links in the header to find what you need. To request help or report a bug, please [open an issue](https://github.com/xmtp/docs-xmtp-org/issues). We know how important it is to have reliable documentation and we appreciate your patience. ## Cursors with XMTP This document explains the concept of **cursors** as they relate to message synchronization on the XMTP network. Cursors are a fundamental part of how XMTP clients efficiently fetch new messages and maintain state, particularly with the `sync()` family of functions. While cursors are managed automatically by the XMTP SDKs, understanding how they work is crucial for debugging and for grasping the underlying mechanics of message synchronization. ### What is a cursor? A cursor is a pointer or a marker that an XMTP client maintains for each topic it subscribes to (both [group message topics](/protocol/topics#group-message-topic) and [welcome message topics](/protocol/topics#welcome-message-topic)). This cursor is stored locally and is specific to each app installation. Think of it as a bookmark in the chronological log of messages and events for a given topic. Its purpose is to remember the exact point up to which an installation has successfully synchronized its data. ### How cursors work with `sync()` The primary role of a cursor becomes evident when you use the `sync()` functions (`conversation.sync()`, `conversations.sync()`, and `conversations.syncAll()`). 1. **Initial sync**: The first time an app installation calls `sync()` for a specific conversation, it fetches all available messages and events from the network for that conversation's topic. 2. **Cursor placement**: Once the sync is complete, the SDK places a cursor at the end of that batch of fetched messages. 3. **Subsequent syncs**: On the next `sync()` call for that same conversation, the client sends its current cursor position to the network. The network then returns only the messages and events that have occurred *after* that cursor. 4. **Cursor advancement**: After the new messages are successfully fetched, the SDK advances the cursor to the new latest point. This process ensures that each `sync()` call only retrieves what's new, making synchronization efficient by avoiding the re-downloading of messages the client already has. #### How Cursors Enable Efficient Sync The XMTP SDKs use cursors to make message synchronization highly efficient. The design principle is to fetch new data from the network with `sync()` while providing access to historical data from a local database. * **`sync()` fetches new data from the network:** The `sync()` functions are designed specifically to retrieve new messages and events from the network. To do this efficiently, the SDK advances the cursor to the position of the last synchronized item. On subsequent `sync()` calls, the client provides this cursor, and the network returns only what has arrived since. This forward-only cursor movement is an intentional design choice that prevents re-downloading data the client already has. * **Access old messages from the local database:** Once `sync()` fetches messages from the network, they are stored in a local database managed by the SDK. You can query this database at any time to retrieve historical messages without making a network request. This provides fast, local access to the full message history available to the installation. * **History on new devices is handled by history sync:** The behavior of cursors should not be confused with loading message history on a new device. A new app installation lacks the encryption keys to decrypt old messages. Even if it could fetch them from the network, they would be unreadable. [History sync](/chat-apps/list-stream-sync/history-sync) is the dedicated process for securely transferring message history and the necessary encryption keys to a new installation. * **Streaming does not affect the cursor:** Receiving messages via a real-time `stream()` does not move the cursor. Streaming provides instant message delivery but doesn't guarantee order or completeness if the client is briefly offline. `sync()` serves as the mechanism to ensure the local state is complete and correctly ordered, and only then is the cursor advanced. ### Cursors for different sync functions Each `sync()` function corresponds to a different type of cursor: * `conversation.sync()`: This operates on the **group message topic** for a single conversation. It moves the cursor for that specific conversation, fetching new messages or group updates (like name changes). * `conversations.sync()`: This operates on the **welcome message topic**. It moves the cursor for welcome messages, fetching any new conversations the user has been invited to. It does *not* fetch the contents of those new conversations. * `conversations.syncAll()`: This is the most comprehensive sync. It effectively performs the actions of the other two syncs for all of the user's conversations. It moves the cursors for the welcome topic *and* for every individual group message topic, ensuring the client has fetched all new conversations and all new messages in existing conversations. For example, here is a sequence diagram illustrating how cursors operate with `conversation.sync()`: ![Sequence diagram showing cursor flow during conversation.sync() operation, illustrating how cursors track message positions and enable incremental synchronization](https://raw.githubusercontent.com/xmtp/docs-xmtp-org/refs/heads/main/docs/pages/img/cursor-flow.png) By understanding cursors, you can better reason about the behavior of your app's synchronization logic and the data being transferred from the XMTP network. ## Envelope types with XMTP This document covers **envelope types** that clients can publish to XMTP APIs. These are the top-level message structures that can be sent and received through the XMTP network. This information is primarily useful for: * Developers contributing to the XMTP protocol itself * Understanding XMTP internals and debugging * Reading XMTP Improvement Proposals (XIPs) The envelope types described here are handled automatically by XMTP SDKs and rarely need direct interaction from app developers. For **app development**, you'll typically work with [content types](/chat-apps/content-types/content-types) instead. Content types define how your app's messages are structured and encoded (text, attachments, reactions, etc.) and are what you'll use in your day-to-day development. ### Overview XMTP supports several envelope types that clients can publish to the network: * **Group message envelopes**: Contain MLS protocol messages (application messages and commits) * **Welcome message envelopes**: Bootstrap new members into existing groups * **Key package envelopes**: Register cryptographic credentials for user installations * **Identity update envelopes**: Verify and authenticate user identities These envelope types work together to enable secure group communication with [forward secrecy and post-compromise security](/protocol/security). **Note**: In the MLS context, "group" refers to any collection of clients that share cryptographic state, which includes both direct message (1:1) or group chat conversations. ### Group message envelope A group message envelope contains an MLS protocol message that can be either an application message or a commit message. This is the primary envelope type used for day-to-day communication. #### Application messages Application messages represent the actual content that users send to each other, including: * Text messages * Attachments * Reactions * Read receipts * On-chain transaction references * Custom content types Application messages are: * Encrypted using the group's shared secrets * Authenticated with sender signatures * Encoded using XMTP content types * Sent on the `groupMessage` topic #### Commit messages Commit messages update the group's cryptographic state, membership, and permissions. While technically a single message type, commit messages serve many different purposes. Understanding commit messages is especially helpful when debugging and understanding your app's user experience. For example: * Different commit types help explain why certain messages appear in logs * Some commits happen invisibly, while others are tied to user actions ##### User-initiated commits * **Add member commits**: When a user explicitly adds someone to a group * **Remove member commits**: When a user removes someone from a group * **Update metadata commits**: When a user changes group name, description, or permissions * **Update permissions commits**: When a user modifies group permission settings ##### System-initiated commits * **Key update commits**: Automatically generated when a new member joins before sending their first message * **Missing member commits**: Triggered when the system detects someone is missing from the group * **Scheduled commits**: Periodic commits for security maintenance ##### MLS protocol commits * **Update path commits**: Generated by the MLS protocol for key rotation and security * **External sender commits**: For handling external participants For more information about MLS commits, see [RFC 9420 Section 12.4](https://www.rfc-editor.org/rfc/rfc9420.html#section-12.4). ### Welcome message envelope A welcome message envelope bootstraps a new member into an existing group. The welcome is dependent on the newcomer's [key package](#key-package-envelope) and provides the new member with the current state of the group after application of a [commit message](#commit-messages). A welcome message contains: * Group context information * Encrypted group secrets * Tree structure for the group * Confirmation tags for epoch verification If decryption fails due to an outdated or missing key package, the SDK automatically fetches the latest package and retries. ### Key package envelope A key package envelope registers cryptographic credentials for a user installation. XMTP SDKs create and upload fresh key packages behind the scenes when an installation is initialized or rotated. Think of a key package as a calling card for an installation that says: here's how to encrypt to me, here's how long this card is valid, and it's signed so you can trust it. A key package contains: * Public key for encrypting welcome messages * Signature key for authentication * Capabilities (supported protocol version, cipher suites, lifetime, etc.) * Credential for identity verification * Content of the leaf node representing this client Group members cache key packages to authenticate future handshakes and welcome material, enabling asynchronous addition of clients to groups. When an app inspects a group member, the SDK provides a `PublicIdentity` object containing the decoded fields, allowing apps to display identity information or check if the key package has expired. ### Identity update envelope An identity update envelope verifies and authenticates a user identity across the network. It is signed by the user's identity and verifiable by peers. Identity update envelopes enable group members to rotate their signature or HPKE keys while preserving group continuity and authenticity. They handle: * Linking an installation to an XMTP inbox * Key rotation and revocation * Linking multiple devices (though [history sync](/chat-apps/list-stream-sync/history-sync) is used to synchronize data between those devices) Identity update messages are stored permanently to ensure continuity of trust and identity verification. ## Epochs with XMTP With XMTP, each [commit](/protocol/envelope-types#commit-messages) starts a new epoch, which represents the current cryptographic state of a group chat. Epochs are a core concept from [Messaging Layer Security](https://messaginglayersecurity.rocks/) (MLS), which XMTP implements for secure group messaging. Epochs work according to these requirements: * Sequential numbering: Epochs are strictly ordered and increase by one with each commit (epoch 1, epoch 2, etc.). * New keys: Each epoch introduces a fresh encryption key, and keys from previous epochs are discarded (with the exception of a configurable number of recent past epochs that may be retained for special cases). * Decryption requirement: To read messages or commits in a given epoch, a member must have the correct epoch key. * Fork risk: If members diverge on which epoch they're in (e.g., due to missed or out-of-order commits), they won't be able to decrypt each other's messages, causing a fork. [Intents](/protocol/intents) help ensure two types of ordering: * **Epoch ordering**: Commits must be published in the correct epoch (one greater than the last applied epoch) to be processed by clients * **Consistent ordering**: All clients must receive published commits in the same order to prevent forks, regardless of epoch validity Intents achieve epoch ordering by enabling retries, while relying on the server's guarantee of consistent ordering across all clients. ### Handle concurrent commits When multiple commits arrive at nearly the same time, XMTP uses a "first-to-arrive wins" approach. For example, if commits 2 and 3 both attempt to advance from epoch 1, whichever commit arrived first becomes the accepted next epoch, and the other commit is rejected. For example, if commit 2 arrived first, it will be built on epoch 2 and commit 3 will be rejected. However, [intents](/protocol/intents) provide a mechanism for the rejected commit 3 to be retried. The intent can be reprocessed against the new epoch 2 state, allowing the operation to succeed in the updated context rather than being permanently lost. ## Identity model with XMTP XMTP's identity model includes an inbox ID and its associated identities and installations. With an inbox ID at its core, instead of a specific wallet address or other identity value, the model is designed for extensibility. A user can associate any number of identity types to their inbox ID and use them to send and receive messages. This gives the user the freedom to add and remove identity types as they naturally change over time, while maintaining the same stable inbox destination for their messages. ![Diagram showing a core inbox ID associated with multiple identities, with access to multiple apps](https://raw.githubusercontent.com/xmtp/docs-xmtp-org/refs/heads/main/docs/pages/img/multi-id.png) The identity model also allows XMTP to support any identity type, as long as it can produce a verifiable cryptographic signature. Currently supported identity types include Ethereum EOAs, Ethereum smart contract wallets, and passkeys. ### Inbox ID An **inbox ID** is a user's stable destination for their messages. Their inbox ID remains constant even as they add or remove [identities](#identity) and [installations](#installation). The inbox ID is derived from the hash of the first associated wallet address and a nonce and acts as an opaque identifier that apps use for messaging. ### Identity An **identity** is an addressable account that can be associated with an inbox ID. Each identity has a type (like EOA, smart contract wallet, or passkey) and an identifier (like an Ethereum address). * Multiple identities can be linked to a single inbox ID * The first identity becomes the **recovery identity** with special privileges * All messages sent to any associated identity are delivered to the same inbox * Any identity that can produce a verifiable cryptographic signature can be supported by XMTP ### Installation An **installation** represents a specific app installation that can access an inbox. Each installation has its own cryptographic keys for signing messages and participating in conversations. * Generated automatically when `Client.create()` is called for the first time with an identity that hasn't been used with XMTP before * Multiple installations can access the same inbox (up to 10) * Installations can be revoked by the recovery identity ### Relationships **One inbox ID** → **multiple identities**: Users can receive messages as any of their identities, all flowing to the same inbox ```text Inbox ID (stable destination for messages) ├── Identity 1 (recovery identity, first identity added to an inbox) ├── Identity 2 (EOA wallet) ├── Identity 3 (SCW wallet) └── Any identity that can produce a verifiable cryptographic signature ``` **One identity** → **multiple installations**: Users can access their messages from different apps on the same or different devices ```text Each identity can authenticate new installations: ├── Installation A (phone app) ├── Installation B (web app) ├── Installation C (desktop app) └── Up to 10 installations ``` ### Identity actions To learn how to build agents with identity actions, see [Manage agent local database files and installations](/agents/build-agents/local-database). To learn how to build chat apps with identity actions, see [Manage XMTP inboxes, identities, and installations](/chat-apps/core-messaging/manage-inboxes). ## Intents with XMTP Intents provide an internal state machine, or "bookkeeping" mechanism, for reliably applying changes to XMTP group chat states, even when the process encounters retries, crashes, race conditions, or ordering issues. Developers building apps and agents with XMTP don't need to work with intents directly, but understanding them provides insight into how the protocol maintains integrity behind the scenes. ### Intent actions An intent represents an action that intends to change the state of a group chat via a [commit](/protocol/envelope-types#commit-messages), along with enough information to retry the action if it fails. Each commit rotates the group's encryption state into a new [epoch](/protocol/epochs) and must be applied in epoch order. If a client processes a commit that is in an incorrect epoch, it will simply discard the commit. Intents provide a structured way to track the multi-step process of publishing commits, handling retries, and recovering from interruptions. This ensures that every commit is eventually published in the correct epoch. Examples of intent actions include: * Add member: Add a participant to a group * Remove member: Remove a participant from a group * Send message: Deliver an application message to the group * Change metadata: Rename a group, for example ### Intent states Each intent progresses through a series of states as it is processed: * **To publish**: Intent has been created and queued, but not yet sent * **Published**: Commit has been sent to the network * **Error**: Intent failed with a permanent, non-retryable error (for example, a member without adequate permission tries to add a member, or the member was removed from the group). These intents will not be retried. * Note: Temporary, retryable failures (such as network issues or app restarts) keep the intent in the **To publish** state for retry on the next sync. * **Committed**: Commit has been accepted into the group's state, and dependent operations can now be performed. For example, after adding group members, welcome messages can be sent with the new encryption state. * **Processed**: Intent is fully complete and all related operations have finished By tracking intent states, XMTP ensures that if an app crashes before a commit has been accepted, for example, the commit process can resume later from the stored state without losing intent information. ### Example intent flow ![Flow diagram showing the progression of an intent through different states: Pending → Committed → Processed, illustrating how XMTP tracks and manages group operations](https://raw.githubusercontent.com/xmtp/docs-xmtp-org/refs/heads/main/docs/pages/img/intents-flow.png) ## XMTP protocol overview XMTP is a decentralized messaging protocol that enables secure, end-to-end encrypted communication between any identities that can produce a verifiable cryptographic signature. XMTP implements [Messaging Layer Security](https://messaginglayersecurity.rocks/) (MLS), which is designed to operate within the context of a messaging service. As the messaging service, XMTP needs to provide two services to facilitate messaging using MLS: * An [authentication service](https://messaginglayersecurity.rocks/mls-architecture/draft-ietf-mls-architecture.html#name-authentication-service) * A [delivery service](https://messaginglayersecurity.rocks/mls-architecture/draft-ietf-mls-architecture.html#name-delivery-service) This section covers the elements of XMTP that provide these services. :::info[Who should read these docs] This protocol documentation is designed for: * Protocol contributors working on XMTP's core implementation * Security researchers auditing XMTP's cryptographic design * Anyone curious about the technical details behind XMTP's messaging For most developers, the [Build chat apps](/chat-apps/intro/get-started) and [Build agents](/agents/get-started/build-an-agent) sections provide the practical guidance needed to build with XMTP. ::: ### Encryption The encryption elements are mainly defined by MLS, with some additions by XMTP. To learn more, see: * [Security](/protocol/security) XMTP and MLS prioritize security, privacy, and message integrity through advanced cryptographic techniques, delivering end-to-end encryption for both 1:1 and group conversations * [Epochs](/protocol/epochs) Represent the cryptographic state of a group at any point in time. Each group operation (like adding members) creates a new epoch with fresh encryption keys * [Envelope types](/protocol/envelope-types) Messages are packaged as envelope types that contain the actual message data plus metadata for routing and processing. ### Identity The identity elements are mainly defined by XMTP. To learn more, see: * [Inboxes, identities, and installations](/protocol/identity) The identity model includes an inbox ID and its associated identities and installations. * [Wallet signatures](/protocol/signatures) Authenticate users using verifiable cryptographic signatures. ### Delivery The delivery elements are mainly defined by XMTP. To learn more, see: * [Topics](/protocol/topics) Messages are routed through topics, which are unique addresses that identify conversation channels. * [Cursors](/protocol/cursors) Enable efficient message synchronization by tracking where each client left off when fetching new messages. * [Intents](/protocol/intents) Provide reliable groupstate management through an internal bookkeeping system that handles retries, crashes, and race conditions when applying group changes. ### Protocol evolution XMTP evolves through **[XMTP Improvement Proposals](/protocol/xips)** (XIPs), which are design documents that propose new features and improvements. This governance process ensures systematic and decentralized protocol development. ### Additional resources For a broader vision of XMTP's approach to core concepts, see the following topics on [xmtp.org](https://xmtp.org): * [Security](https://xmtp.org/security) * [Identity](https://xmtp.org/identity) * [Digital money](https://xmtp.org/digital-money) * [Consent](https://xmtp.org/consent) * [Decentralizing XMTP](https://xmtp.org/decentralization) ## Messaging security properties with XMTP XMTP delivers end-to-end encrypted 1:1 and group chat using the following resources: * Advanced cryptographic techniques * Secure key management practices * MLS ([Messaging Layer Security](https://www.rfc-editor.org/rfc/rfc9420.html)) Specifically, XMTP messaging provides the comprehensive security properties covered in the following sections. In these sections, **group** refers to the MLS concept of a group, which includes both 1:1 and group conversations. 🎥 **walkthrough: XMTP and MLS** This video provides a walkthrough of XMTP's implementation of MLS.