RFC 9404: JSON Meta Application Protocol (JMAP) Blob Management Extension
- B. Gondwana, Ed.
Abstract
The JSON Meta Application Protocol (JMAP) base protocol (RFC 8620) provides the ability to upload and download arbitrary binary data via HTTP POST and GET on a defined endpoint. This binary data is called a "blob".¶
This extension adds additional ways to create and access blobs by making inline method calls within a standard JMAP request.¶
This extension also adds a reverse lookup mechanism to discover where blobs are referenced within other data types.¶
Status of This Memo
This is an Internet Standards Track document.¶
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.¶
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
https://
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://
1. Introduction
Sometimes JMAP [RFC8620] interactions require creating a blob and then referencing it. In the same way that IMAP literals were extended by [RFC7888], embedding small blobs directly into the JMAP method calls array can be an option for reducing round trips.¶
Likewise, when fetching an object, it can be useful to also fetch the raw content of that object without a separate round trip.¶
Since raw blobs may contain arbitrary binary data, this document defines a use of the base64 coding specified in [RFC4648] for both creating and fetching blob data.¶
When JMAP is proxied through a system that applies additional access restrictions, it can be useful to know which objects reference any particular blob; this document defines a way to discover those references.¶
2. Conventions Used in This Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The definitions of JSON keys and datatypes in the document follow the conventions described in [RFC8620].¶
3. Addition to the Capabilities Object
The capabilities object is returned as part of the JMAP Session object; see [RFC8620], Section 2.¶
This document defines an additional capability URI.¶
3.1. urn:ietf:params:jmap:blob
The presence of the capability urn in the
account
The value of this property in the JMAP Session capabilities property MUST be an empty object.¶
The value of this property in an account's account
4. Blob Methods
A blob is a sequence of zero or more octets.¶
JMAP [RFC8620] defines the
Blob/copy method, which is unchanged by this specification and is
selected by the urn capability.¶
The following JMAP methods are selected by the
urn capability.¶
4.1. Blob/upload
This is similar to a Foo/set in [RFC8620] in some
ways. However, blobs cannot be updated or deleted, so only create is
allowed in the method call. Also, blobs do not have state, so there is no
state field present in the method response.¶
Parameters¶
Result¶
The result is the same as for Foo/set in [RFC8620], with created and notCreated objects
mapping from the creation id.¶
The created objects contain:¶
The created objects will also contain any other properties identical to those that would be returned in the JSON response of the upload endpoint described in [RFC8620]. This may be extended in the future; in this document, it is anticipated that implementations will extend both the upload endpoint and the Blob/upload responses in the same way.¶
If there is a problem with a creation, then the server will return a
notCreated response with a map from the failed creation id to a
SetError object.¶
For each successful upload, servers MUST add an entry to the
createdIds map ([RFC8620], Section 3.3) for the request; even if the caller did not explicitly pass a
createdIds, the value must be available to later methods defined in the same
Request Object. This allows the blobId to be used via back-reference in
subsequent method calls.¶
The created blob will have the same lifetime and same expiry semantics as any other binary object created via the mechanism specified in [RFC8620], Section 6.¶
Uploads using this mechanism will be restricted by the maxUploadSize limit for JMAP requests specified by the server, and clients SHOULD consider using the upload mechanism defined by [RFC8620] for blobs larger than a megabyte.¶
UploadObject¶
DataSourceObject¶
Exactly one of:¶
or a blobId source:¶
If null, then offset is assumed to be zero.¶
If null, then length is the remaining octets in the blob.¶
If the range cannot be fully satisfied (i.e., it begins or extends past
the end of the data in the blob), then the Data
If the data properties have any invalid references or invalid data contained in them, the server MUST NOT guess the user's intent and MUST reject the creation and return a notCreated response for that creation id.¶
Likewise, invalid characters in the base64 of data:asBase64 or invalid UTF-8 in data:asText MUST result in a notCreated response.¶
It is envisaged that the definition for Data
A server MUST accept at least 64 Data
4.1.1. Blob/upload Simple Example
The data:asBase64 field is set over multiple lines for ease of publication here; however, the entire data:asBase64 field would be sent as a continuous string with no wrapping on the wire.¶
Method Call:¶
Response:¶
4.2. Blob/get
A standard JMAP get, with two additional optional parameters:¶
Request Properties:¶
Any of:¶
If not given, the properties default to data and size.¶
Result Properties:¶
The size value MUST always be the number of octets in the underlying blob, regardless of offset and length.¶
The data fields contain a representation of the octets within the selected
range that are present in the blob. If the octets selected are not valid
UTF-8 (including truncating in the middle of a multi-octet sequence)
and data or data:asText was requested, then the key is
MUST be set to true, and the data:asText response value MUST be null.
In the case where data was requested and the data is not valid UTF-8,
then data:asBase64 MUST be returned.¶
If the selected range requests data outside the blob (i.e., the
offset+length is larger than the blob), then the result is either just the
octets from the offset to the end of the blob or an empty string if the
offset is past the end of the blob. Either way, the isTruncated
property in the result MUST be set to true to tell the
client that the requested range could not be fully satisfied. If digest was
requested, any digest is calculated on the octets that would be
returned for a data field.¶
Servers SHOULD store the size for blobs in a format that is efficient to read, and clients SHOULD limit their request to just the size parameter if that is all they need, as fetching blob content could be significantly more expensive and slower for the server.¶
4.2.1. Blob/get Simple Example
In this example, a blob containing the string "The quick brown fox jumped over
the lazy dog." has blobId Gc0854fb9fb03c41.¶
The first method call requests just the size for multiple blobs, and the second requests both the size and a short range of the data for one of the blobs.¶
Method Calls:¶
Responses:¶
4.3. Blob/lookup
Given a list of blobIds, this method does a reverse lookup in each of the provided type names to find the list of Ids within that data type that reference the provided blob.¶
Since different datatypes will have different semantics of "contains", the definition of "reference" is somewhat loose but roughly means "you could discover this blobId by looking at this object or at other objects recursively contained within this object".¶
For example, with a server that supports [RFC8621], if a Mailbox references a blob and if any Emails within that Mailbox reference the blobId, then the Mailbox references that blobId. For any Thread that references an Email that references a blobId, it can be said that the Thread references the blobId.¶
However, this does not mean that if an Email references a Mailbox in its mailboxIds property, then any blobId referenced by other Emails in that Mailbox are also referenced by the initial Email.¶
Parameters¶
Response¶
BlobInfo Object¶
If a blob is not visible to a user or does not exist on the server at all, then the server MUST still return an empty array for each type as this doesn't leak any information about whether the blob is on the server but not visible to the requesting user.¶
5. Security Considerations
All security considerations for JMAP [RFC8620] apply to this specification. Additional considerations specific to the data types and functionality introduced by this document are described here.¶
JSON parsers are not all consistent in handling non-UTF-8 data.
JMAP requires that all JSON data be UTF-8 encoded, so servers
MUST only return a null value if data:asText is
requested for a range of octets that is not valid UTF-8 and set
is.¶
Servers MUST apply any access controls, such that if the authenticated user would be unable to discover the blobId by making queries, then this fact cannot be discovered via a Blob/lookup. For example, if an Email exists in a Mailbox that the authenticated user does not have access to see, then that emailId MUST NOT be returned in a lookup for a blob that is referenced by that email.¶
The server MUST NOT trust that the data given to a Blob/upload is a well-formed instance of the specified media type. Also, if the server attempts to parse the given blob, only hardened parsers designed to deal with arbitrary untrusted data should be used. The server SHOULD NOT reject data on the grounds that it is not a valid specimen of the stated type.¶
With carefully chosen data sources, Blob/upload can be used to recreate dangerous content on the far side of security scanners (anti-virus or exfiltration scanners, for example) that may be watching the upload endpoint. Server implementations SHOULD provide a hook to allow security scanners to check the resulting blob after concatenating the data sources in the same way that they do for the upload endpoint.¶
Digest algorithms can be expensive for servers to calculate. Servers that share resources between multiple users should track resource usage by clients and rate-limit expensive operations to avoid resource starvation.¶
6. IANA Considerations
6.1. JMAP Capability Registration for "blob"
IANA has registered the "blob" JMAP capability as follows:¶
6.2. JMAP Error Codes Registration for "unknownDataType"
IANA has registered the "unknown
6.3. Creation of "JMAP Data Types" Registry
IANA has created a new registry called "JMAP Data Types". Table 1 shows the initial contents of this new registry.¶
The registration policy for this registry is "Specification Required" [RFC8126]. Either an RFC or a similarly stable reference document defines a JMAP Data Type and associated capability.¶
IANA will appoint designated experts to review requests for additions to this registry, with guidance to allow any registration that provides a stable document describing the capability and control over the URI namespace to which the capability URI points.¶
7. References
7.1. Normative References
- [RFC2119]
-
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10
.17487 , , <https:///RFC2119 www >..rfc -editor .org /info /rfc2119 - [RFC3230]
-
Mogul, J. and A. Van Hoff, "Instance Digests in HTTP", RFC 3230, DOI 10
.17487 , , <https:///RFC3230 www >..rfc -editor .org /info /rfc3230 - [RFC4648]
-
Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10
.17487 , , <https:///RFC4648 www >..rfc -editor .org /info /rfc4648 - [RFC8174]
-
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10
.17487 , , <https:///RFC8174 www >..rfc -editor .org /info /rfc8174 - [RFC8620]
-
Jenkins, N. and C. Newman, "The JSON Meta Application Protocol (JMAP)", RFC 8620, DOI 10
.17487 , , <https:///RFC8620 www >..rfc -editor .org /info /rfc8620
7.2. Informative References
- [RFC7888]
-
Melnikov, A., Ed., "IMAP4 Non
-synchronizing , RFC 7888, DOI 10Literals" .17487 , , <https:///RFC7888 www >..rfc -editor .org /info /rfc7888 - [RFC8126]
-
Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10
.17487 , , <https:///RFC8126 www >..rfc -editor .org /info /rfc8126 - [RFC8621]
-
Jenkins, N. and C. Newman, "The JSON Meta Application Protocol (JMAP) for Mail", RFC 8621, DOI 10
.17487 , , <https:///RFC8621 www >..rfc -editor .org /info /rfc8621 - [RFC9007]
-
Ouazana, R., Ed., "Handling Message Disposition Notification with the JSON Meta Application Protocol (JMAP)", RFC 9007, DOI 10
.17487 , , <https:///RFC9007 www >..rfc -editor .org /info /rfc9007
Acknowledgements
Joris Baum, Jim Fenton, Neil Jenkins, Alexey Melnikov, Ken Murchison, Robert Stepanek, and the JMAP Working Group in the IETF.¶