On Stable Storage for Items in Concise Binary Object Representation (CBOR)Sandelman Software Worksmcr+ietf@sandelman.caUniversität Bremen TZIPostfach 330440D-28359 BremenGermany+49-421-218-63921cabo@tzi.org
art
cbormagic numberfile identificationThis document defines a stored ("file") format for Concise Binary
Object Representation (CBOR) data items
that is friendly to common systems that recognize file types, such
as the Unix file(1) command.Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by
the Internet Engineering Steering Group (IESG). Further
information on Internet Standards is available in Section 2 of
RFC 7841.
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
() in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Revised BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Revised BSD License.
Table of Contents
. Introduction
. Terminology
. Requirements for a Magic Number
. Protocol
. The CBOR-Protocol-Specific Tag
. Enveloping Method: CBOR Tag Wrapped
. Example
. Enveloping Method: Labeled CBOR Sequence
. Example
. Security Considerations
. IANA Considerations
. Labeled CBOR Sequence Tag
. CBOR-Labeled Non-CBOR Data Tag
. CBOR Tags for CoAP Content-Format Numbers
. References
. Normative References
. Informative References
. Advice to Protocol Designer
. Is the on-wire format new?
. Can many items be trivially concatenated?
. Are there tags at the start?
. CBOR Tags for CoAP Content Formats
. Content-Format Tag Examples
. Example from Openswan
. Using CBOR Labels for Non-CBOR Data
. Content-Format Tag Examples
Acknowledgements
Contributors
Authors' Addresses
IntroductionSince very early in computing, operating systems have sought ways to mark which files could be processed by which programs.
In Unix, everything is a stream of bytes; identifying the contents of a stream of bytes became a heuristic activity.For instance, the Unix file(1) command, which has existed since 1973 , has been able to identify many file formats based upon the contents of the file for decades.Many systems (Linux, macOS, Windows) will select the correct application based upon the file contents if the system cannot determine it by other means.
For instance, in classical Mac OS, a resource fork was maintained
separately from the file data that included file type information;
this way, the OS ideally never needed to know anything about the file
data contents to determine the media type.Many other systems do this by using file extensions.
Many common Web servers derive the media-type information from file extensions.Having a media type associated with the file contents can avoid some
of the brittleness of this approach.
When files become disconnected from their type information, such as when attempting to do forensics on a damaged system, being able to identify the type of information stored in a file can become very important. A common way to identify the type of a file from its contents is to
place a "magic number" at the start of the file contents . In
the media type registration template , a
magic number is asked for, if available, as is a file extension.
A challenge for the file(1) command is often that it can be confused
by recognizing the overall encoding but not the content being encoded. For instance, an Android Package Kit APK (as used to transfer and store an application) may be identified as a ZIP file. Additionally, both OpenOffice and MSOffice files are ZIP files
of XML files; the identification may stop at identifying them as ZIP files.
As CBOR becomes a more and more common encoding for a wide variety of artifacts, identifying them as just "CBOR" is probably not sufficient.
This document provides a way to encode a magic number into the beginning of a CBOR format file.
As a CBOR format may use a single CBOR data item or a CBOR sequence of
data items ,
two possible methods of enveloping data are presented; a CBOR Protocol designer will specify one.
(A CBOR Protocol is a specification that uses CBOR as its encoding.)This document also gives advice to designers of CBOR Protocols on
choosing one of these mechanisms for identifying their contents.
This advice is informative.A third method is also proposed by which a CBOR format tag is prepended to identify non-CBOR files.
Further information on this method appears in because it is not
about identifying media types containing CBOR-encoded data items.
This includes a simple way to derive a magic number for
content-formats as defined in , even if the file is
not in CBOR form.
Examples of CBOR Protocols currently under development include Concise
Software Identification Tags and
Entity Attestation Tokens .
CBOR Object Signing and Encryption (COSE) itself is considered infrastructure.
The encoding of public keys in CBOR as C509 as described in would benefit from being an identified CBOR Protocol.A major inspiration for this document is observing the disarray in
certain ASN.1-based systems where most files are Privacy-Enhanced Mail (PEM) encoded;
these files are all identified by the
extension "pem", which confounds public keys, private keys,
certificate requests, and S/MIME content.While the envelopes defined in this specification add information to how data conforming to CBOR Protocols are stored in files, there is no requirement that either type of envelope be transferred on the wire.
However, there are some protocols that may benefit from having such a magic number on the wire if they are presently using a different (legacy) encoding scheme.
The presence of the identifiable magic sequence can be used to signal
that a CBOR Protocol is being used as opposed to a legacy scheme.TerminologyByte is a synonym for octet.
The term "byte string" refers to the data item defined in .The term "file" is understood to stand in a general way for a stored
representation that is somewhat detached from the original context of
usage of that representation; its usage in this document encompasses
similar units of storage that may have different identification
schemes such as partitions or media blocks.The term "diagnostic notation" refers to the human-readable notation
for CBOR data items defined in and .The term "CDDL" (Concise Data Definition Language) refers to the
language defined in .The function TN(ct) is defined in .Requirements for a Magic NumberIdeally, a magic number is a fingerprint that is unique to a specific CBOR Protocol, is present in the first few (small multiple of 4) bytes of the file and does not change when the contents change, and does not depend upon the length of the file.Less ideal solutions have a pattern that needs to be matched, but in which some bytes need to be ignored.
While the Unix file(1) command can be told to ignore certain bytes, this can lead to ambiguities.ProtocolThis section presents two enveloping methods.
Both use CBOR tags in a way that results in a deterministic first 8 to 12 bytes.
The Protocol designer determines which one to use; see for
some guidance.The CBOR-Protocol-Specific TagIn both enveloping methods,
CBOR Protocol designers need to obtain a CBOR tag for each kind of object that they might store in files.
As there are more than 4 billion available 4-byte tags, there should be little issue in allocating a few to each available CBOR Protocol.The IANA policy for 4-byte CBOR tags is First Come First Served so only a simple interaction (e.g., via Web or email) with IANA is required. The interaction includes filling in the small template provided in .
In the template, a reference to this
specification (RFC 9277) alongside the Description of
semantics is suggested.Allocation of the CBOR tag needs to be initiated by the designer of the CBOR
Protocol, who can provide a proposed tag number.
In order to be in the 4-byte range, and so that there are no
leading zero bytes in the 4-byte encoding of the tag number, the
value needs to be in the range 0x01000000 (decimal
16777216) to 0xFFFFFFFF (decimal 4294967295) inclusive.
It is further suggested to avoid values that have an embedded zero byte in
the 4 bytes of their binary representation (such as 0x12003456), as
these may confuse implementations that treat the magic number as a C string.The use of a sequence of four ASCII codes which are mnemonic to the
protocol is encouraged, but not required (there may be reasons to
encode other information into the tag; see for an example).
For instance, uses "OPSN", which translates to the tag
number 1330664270 registered for it.In , the Constrained Application Protocol (CoAP) defines the "CoAP Content-Formats" registry to assign Content-Format Numbers () to Content Types in a specific Content Coding.
For CBOR data items that form a representation that is already described by such a Content-Format
Number, a tag number has proactively been allocated
in (see for details and examples).
Enveloping Method: CBOR Tag WrappedThe CBOR Tag Wrapped method is appropriate for use with CBOR Protocols that encode a single CBOR data item.
This data item is enveloped into two nested tags:The outer tag is a self-described CBOR tag, 55799, as
described in .The tag content of the outer tag is a second CBOR tag whose tag number has
been allocated to describe the specific Protocol involved, as
discussed in .
The tag content of this inner tag is the single CBOR data item.This method wraps the CBOR data item as CBOR tags usually do.
Applications that need to send the stored CBOR data item across a
constrained network may wish to remove the two tags if the type is
understood from the protocol context, e.g., from a CoAP Content-Format
Option ().
Therefore, a CBOR Protocol specification may pick the specific cases
where the CBOR Tag Wrapped enveloping method is to be used.
For instance, it might specify its use for storing the representation
in a local file or for Web access, but not within protocol messages that
already provide the necessary context.ExampleTo construct an example without registering a new tag, we use
the Content-Format ID assigned for application/senml+cbor (112)
of the "CoAP Content-Formats" registry ).Using the technique described in , this translates into the
tag TN(112) = 1668546929.With this tag, the SenML-CBOR pack [{0: "current", 6: 3, 2: 1.5}] would be enveloped as follows (in diagnostic notation):
55799(1668546929([{0: "current", 6: 3, 2: 1.5}]))
Or in hex:
d9 d9f7 # tag(55799)
da 63740171 # tag(1668546929)
81 # array(1)
a3 # map(3)
00 # unsigned(0)
67 # text(7)
63757272656e74 # "current"
06 # unsigned(6)
03 # unsigned(3)
02 # unsigned(2)
f9 3e00 # primitive(15872)
At the representation level, the unique fingerprint for application/senml+cbor is composed of the 8 bytes d9d9f7da63740171 hex, after which the unadorned CBOR data (81... for the SenML data) is appended.Enveloping Method: Labeled CBOR SequenceThe Labeled CBOR Sequence method is appropriate for use with CBOR Sequences as described in .This method prepends a newly constructed, separate data item to the
CBOR Sequence, the label.The label is a nesting of two tags, similar to but distinct from the
CBOR Tag Wrapped methods, with an inner tag content of a constant byte
string.
The total length of the label is 12 bytes.
The outer tag is the self-described CBOR Sequence tag, 55800.
The inner tag is a CBOR tag from the First Come First Served space that uniquely identifies the CBOR Protocol.
As with the CBOR Tag Wrapped method, the use of a 4-byte tag that encodes without zero bytes is encouraged.
The tag content is a 3-byte CBOR byte string containing
0x42_4f_52 ('BOR' in diagnostic notation).
The outer tag in the label identifies the file as being a CBOR Sequence and does so with all
the desirable properties explained in .
Specifically, it does not appear to conflict with any known file types, and it is not valid Unicode in any Unicode encoding.The inner tag in the label identifies which CBOR Protocol is used, as described above.The inner tag content is a constant byte string that is represented as
0x43_42_4f_52, the ASCII characters "CBOR",
which is the CBOR-encoded data item for the 3-byte string
0x42_4f_52 ('BOR' in diagnostic notation).The actual CBOR Protocol data then follows as the next data item(s) in the CBOR Sequence, without a need for any further specific tag.
The use of a CBOR Sequence allows the application to trivially remove the first item with the two tags.Should this file be reviewed by a human (directly in an
editor or in a hexdump display), it will include the ASCII characters "CBOR" prominently.
This value is also included simply because the inner nested tag needs to tag something.ExampleTo construct an example without registering a new tag, we use
ID 272 as assigned for application/missing-blocks+cbor-seq of the "CoAP Content-Formats" registry .Using the technique described in , this translates into the
tag TN(272) = 1668547090.This is a somewhat contrived example, as this is not a media type that
is likely to be committed to storage.
Nonetheless, with this tag, missing blocks list 0, 8, 15 would be
enveloped as (in diagnostic notation):
55800(1668547090('BOR')),
0,
8,
15
Or in hex:
# CBOR sequence with 4 elements
d9 d9f8 # tag(55800)
da 63740212 # tag(1668547090)
43 # bytes(3)
424f52 # "BOR"
00 # unsigned(0)
08 # unsigned(8)
0f # unsigned(15)
At the representation level, the unique fingerprint for
application/missing-blocks+cbor-seq is composed of the 8 bytes
d9d9f8da63740212 hex, after which the unadorned CBOR sequence
(00... for the missing block list given) is appended.Security ConsiderationsThis document provides a way to identify CBOR Protocol objects.
Clearly identifying CBOR contents in files may have a variety of impacts.The most obvious is that it may allow malware to identify interesting
stored objects, and then exfiltrate or corrupt them.Protective applications (that check data) cannot rely on the
applications they try to protect (that use the data) to make
exactly the same decisions in recognizing file formats.
(This is an instance of a check versus use issue.)
For example, end-point assessment technologies should not solely rely
on the labeling approaches described in this document to decide
whether to inspect a given file.
Similarly, depending on operating system configurations and related
properties of the execution environment, the labeling might influence
the default application used to process a file in a way that may not
be predicted by a protective application.
IANA ConsiderationsThese IANA considerations are entirely about CBOR tags in the
"Concise Binary Object Representation (CBOR) Tags" registry . documents the allocation for a
CBOR tag to be used in a CBOR sequence to identify the sequence (an
example for using this tag is found in ). documents the allocation for a CBOR tag to be used in the CBOR-Labeled Non-CBOR Data Enveloping Method (, which also shows examples).
allocates a CBOR tag for each actual or
potential CoAP Content-Format number (examples are in ).Labeled CBOR Sequence TagIANA has allocated tag 55800 for the Labeled CBOR Sequence
Enveloping Method from the "CBOR Tags" registry.
IANA has updated this tag registration to point to this document.This tag is from the First Come First Served area.The value has been picked to have properties similar to the 55799 tag ().The hexadecimal representation of the encoded tag head is 0xd9_d9_f8.This is not valid UTF-8: the first 0xd9 introduces a 3-byte sequence in UTF-8, but the 0xd9 as the second value is not a valid second byte for UTF-8.This is not valid UTF-16: the byte sequence 0xd9d9 (in either endian order) puts this value into the UTF-16 high-half zone, which would signal that this is a 32-bit Unicode value. However, the following 16-bit big-endian value 0xf8_xx is not a valid second sequence according to .
On a little-endian system, it would be necessary to examine the fourth byte to determine if it is valid. That next byte is determined by the subsequent encoding, and has already determined that no valid CBOR encodings result in valid UTF-16.
Data Item:
tagged byte string
Semantics:
indicates that the file contains CBOR Sequences
CBOR-Labeled Non-CBOR Data TagIANA has allocated tag 55801 for the CBOR-Labeled Non-CBOR Data
Enveloping Method () from the "CBOR Tags" registry.
IANA updated this tag registration to point to this document.This tag is from the First Come First Served area.The value has been picked to have properties similar to the 55799 tag ().The hexadecimal representation of the encoded tag head is 0xd9_d9_f9.This is not valid UTF-8: the first 0xd9 introduces a 3-byte sequence in UTF-8, but the 0xd9 as the second value is not a valid second byte for UTF-8.This is not valid UTF-16: the byte sequence 0xd9d9 (in either endian order) puts this value into the UTF-16 high-half zone, which would signal that this is a 32-bit Unicode value. However, the following 16-bit big-endian value 0xf9_xx is not a valid second sequence according to .
On a little-endian system, it would be necessary to examine the fourth byte to determine if it is valid. That next byte is determined by the subsequent encoding, and has already determined that no valid CBOR encodings result in valid UTF-16.
Data Item:
tagged byte string
Semantics:
indicates that the file starts with a CBOR-Labeled Non-CBOR Data label.
CBOR Tags for CoAP Content-Format NumbersIANA allocated the tag numbers 1668546817 (0x63740101) to
1668612095 (0x6374ffff) as follows:
Data Item:
byte string or any CBOR data item (see )
Semantics:
the representation of content-format ct < 65025 is indicated by
tag number
TN(ct) = 0x63740101 + (ct / 255) * 256 + ct % 255
Reference:
RFC 9277
The "CoAP Content-Formats" registry is defined in .ReferencesNormative ReferencesInformation technology -- Programming languages -- CInternational Organization for StandardizationConcise Binary Object Representation (CBOR) SequencesThis document describes the Concise Binary Object Representation (CBOR) Sequence format and associated media type "application/cbor-seq". A CBOR Sequence consists of any number of encoded CBOR data items, simply concatenated in sequence.Structured syntax suffixes for media types allow other media types to build on them and make it explicit that they are built on an existing media type as their foundation. This specification defines and registers "+cbor-seq" as a structured syntax suffix for CBOR Sequences.Concise Binary Object Representation (CBOR)Informative ReferencesCBOR Encoded X.509 Certificates (C509 Certificates)Ericsson ABEricsson ABRISE ABRISE ABNexus Group This document specifies a CBOR encoding of X.509 certificates. The
resulting certificates are called C509 Certificates. The CBOR
encoding supports a large subset of RFC 5280 and all certificates
compatible with the RFC 7925, IEEE 802.1AR (DevID), CNSA, RPKI, GSMA
eUICC, and CA/Browser Forum Baseline Requirements profiles. When
used to re-encode DER encoded X.509 certificates, the CBOR encoding
can in many cases reduce the size of RFC 7925 profiled certificates
with over 50%. The CBOR encoded structure can alternatively be
signed directly ("natively signed"), which does not require re-
encoding for the signature to be verified. The document also
specifies C509 COSE headers, a C509 TLS certificate type, and a C509
file format.
Work in ProgressConcise Software Identification TagsFraunhofer SITNational Security AgencyThe MITRE CorporationNational Institute of Standards and Technology ISO/IEC 19770-2:2015 Software Identification (SWID) tags provide an
extensible XML-based structure to identify and describe individual
software components, patches, and installation bundles. SWID tag
representations can be too large for devices with network and storage
constraints. This document defines a concise representation of SWID
tags: Concise SWID (CoSWID) tags. CoSWID supports a similar set of
semantics and features as SWID tags, as well as new semantics that
allow CoSWIDs to describe additional types of information, all in a
more memory efficient format.
Work in ProgressThe Entity Attestation Token (EAT)Security Theory LLCQualcomm Technologies Inc.Qualcomm Technologies Inc. An Entity Attestation Token (EAT) provides an attested claims set
that describes state and characteristics of an entity, a device like
a phone, IoT device, network equipment or such. This claims set is
used by a relying party, server or service to determine how much it
wishes to trust the entity.
An EAT is either a CBOR Web Token (CWT) or JSON Web Token (JWT) with
attestation-oriented claims. To a large degree, all this document
does is extend CWT and JWT.
Work in Progressfile (command)WikipediaConcise Binary Object Representation (CBOR) TagsIANAConstrained RESTful Environments (CoRE) ParametersIANAarchive (library) file formatBell LabsUnix Programmer's Manual, First Edition: File FormatsASCII format for network interchangeUTF-16, an encoding of ISO 10646This document describes the UTF-16 encoding of Unicode/ISO-10646, addresses the issues of serializing UTF-16 as an octet stream for transmission over the Internet, discusses MIME charset naming as described in [CHARSET-REG], and contains the registration for three MIME charset parameter values: UTF-16BE (big-endian), UTF-16LE (little- endian), and UTF-16. This memo provides information for the Internet community.Media Type Specifications and Registration ProceduresThis document defines procedures for the specification and registration of media types for use in HTTP, MIME, and other Internet protocols. This memo documents an Internet Best Current Practice.The Constrained Application Protocol (CoAP)The Constrained Application Protocol (CoAP) is a specialized web transfer protocol for use with constrained nodes and constrained (e.g., low-power, lossy) networks. The nodes often have 8-bit microcontrollers with small amounts of ROM and RAM, while constrained networks such as IPv6 over Low-Power Wireless Personal Area Networks (6LoWPANs) often have high packet error rates and a typical throughput of 10s of kbit/s. The protocol is designed for machine- to-machine (M2M) applications such as smart energy and building automation.CoAP provides a request/response interaction model between application endpoints, supports built-in discovery of services and resources, and includes key concepts of the Web such as URIs and Internet media types. CoAP is designed to easily interface with HTTP for integration with the Web while meeting specialized requirements such as multicast support, very low overhead, and simplicity for constrained environments.PKCS #1: RSA Cryptography Specifications Version 2.2This document provides recommendations for the implementation of public-key cryptography based on the RSA algorithm, covering cryptographic primitives, encryption schemes, signature schemes with appendix, and ASN.1 syntax for representing keys and for identifying the schemes.This document represents a republication of PKCS #1 v2.2 from RSA Laboratories' Public-Key Cryptography Standards (PKCS) series. By publishing this RFC, change control is transferred to the IETF.This document also obsoletes RFC 3447.Guidelines for Writing an IANA Considerations Section in RFCsMany protocols make use of points of extensibility that use constants to identify various protocol parameters. To ensure that the values in these fields do not have conflicting uses and to promote interoperability, their allocations are often coordinated by a central record keeper. For IETF protocols, that role is filled by the Internet Assigned Numbers Authority (IANA).To make assignments in a given registry prudently, guidance describing the conditions under which new values should be assigned, as well as when and how modifications to existing values can be made, is needed. This document defines a framework for the documentation of these guidelines by specification authors, in order to assure that the provided guidance for the IANA Considerations is clear and addresses the various issues that are likely in the operation of a registry.This is the third edition of this document; it obsoletes RFC 5226.CBOR Object Signing and Encryption (COSE)Concise Binary Object Representation (CBOR) is a data format designed for small code size and small message size. There is a need for the ability to have basic security services defined for this data format. This document defines the CBOR Object Signing and Encryption (COSE) protocol. This specification describes how to create and process signatures, message authentication codes, and encryption using CBOR for serialization. This specification additionally describes how to represent cryptographic keys using CBOR.Sensor Measurement Lists (SenML)This specification defines a format for representing simple sensor measurements and device parameters in Sensor Measurement Lists (SenML). Representations are defined in JavaScript Object Notation (JSON), Concise Binary Object Representation (CBOR), Extensible Markup Language (XML), and Efficient XML Interchange (EXI), which share the common SenML data model. A simple sensor, such as a temperature sensor, could use one of these media types in protocols such as HTTP or the Constrained Application Protocol (CoAP) to transport the measurements of the sensor or to be configured.Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data StructuresThis document proposes a notational convention to express Concise Binary Object Representation (CBOR) data structures (RFC 7049). Its main goal is to provide an easy and unambiguous way to express structures for protocol messages and data formats that use CBOR or JSON.HTTP SemanticsThe Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document describes the overall architecture of HTTP, establishes common terminology, and defines aspects of the protocol that are shared by all versions. In this definition are core protocol elements, extensibility mechanisms, and the "http" and "https" Uniform Resource Identifier (URI) schemes.This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230.Constrained Application Protocol (CoAP) Block-Wise Transfer Options Supporting Robust TransmissionThis document specifies alternative Constrained Application Protocol (CoAP) block-wise transfer options: Q-Block1 and Q-Block2.These options are similar to, but distinct from, the CoAP Block1 and Block2 options defined in RFC 7959. The Q-Block1 and Q-Block2 options are not intended to replace the Block1 and Block2 options but rather have the goal of supporting Non-confirmable (NON) messages for large amounts of data with fewer packet interchanges. Also, the Q-Block1 and Q-Block2 options support faster recovery should any of the blocks get lost in transmission.Information technology - ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER)ITU-TAdvice to Protocol DesignerThis document introduces a choice between wrapping a single CBOR data item into a pair of identifying CBOR tags or prepending an identifying encoded CBOR data item (which, in turn, contains a pair of identifying CBOR tags) to a CBOR Sequence (which might be a single data item).Which should a protocol designer use?In this discussion, one assumes that there is an object stored in a file, perhaps specified by a system operator in a configuration file.For example: a private key used in COSE operations, a public key/certificate in C509 or CBOR format, a recorded sensor reading stored for later transmission, or a COVID-19 vaccination certificate that needs to be displayed in QR code form.Both the Labeled CBOR Sequence and the wrapped tag can be trivially removed by an application before sending the CBOR content out on the wire.The Labeled CBOR Sequence can be slightly easier to remove as, in most cases, CBOR parsers will return it as a unit and then return the actual CBOR item, which could be anything at all and could include CBOR tags that do need to be sent on the wire.On the other hand, having the Labeled CBOR Sequence in the file requires that all programs that expect to examine that file be able to skip what appears to be a CBOR item with two tags nested around a 3-byte byte string.
The 3-byte entry is not of the format the program would normally have processed, so it may be a surprise.
On the other hand, CBOR parsers are generally tolerant of tags that appear: many of them will process extra tags, making unknown tags available as meta information.
A program that is not expecting those tags may just ignore them.As an example of where there was a problem with previous security systems, "PEM" format certificate files grew to be able to contain multiple certificates by simple concatenation.
The PKCS1 format could also contain a private key object followed by one or more certificate objects, but only when in PEM format.
Annoyingly, when in binary DER format (, which like CBOR is self-delimiting), concatenation of certificates was not compatible with most programs as they did not expect to read more than one item in the file.The use of CBOR Tag Wrapped format is easier to retrofit to an existing format with existing and unchangeable stored format for a single CBOR data item.
This new sequence of tags is expected to be trivially ignored by many
existing programs when reading CBOR from files or similar units of storage, even if the program only
supports decoding a single data item (and not a CBOR sequence).
But, a naive program might also then transmit the additional tags across the network.
Removing the CBOR Tag Wrapped format requires knowledge of the two tags involved.
Other tags present might be needed.For a representation matching a specific media-type that is carried in
a CBOR byte string, the byte string head will already have to be
removed for use as such a representation, so it should be easy to
remove the enclosing tag heads as well.
This is of particular interest with the predefined tags provided in
for media types with CoAP Content-Format numbers.Some considerations in the form of survey questions follow.Is the on-wire format new?If the on-wire format is new, then it could be specified with the CBOR Tag Wrapped format if the extra 8 bytes are not a problem.
The stored format is then identical to the on-wire format.If the 8 bytes are a problem on the wire (and they often are if CBOR is being considered), then the Labeled CBOR Sequence format should be adopted for the stored format.Can many items be trivially concatenated?If the programs that read the contents of the file already expect to process all of the CBOR data items in the file (not just the first), then the Labeled CBOR Sequence format may be easily retrofitted.The programs involved may throw errors or warnings on the Labeled CBOR
Sequence if they have not yet been updated, but this may not be a problem.There are situations where multiple objects may be concatenated into a single file.
If each object is preceded by a Labeled CBOR Sequence label, then there may be
multiple such labels in the file.A protocol based on CBOR Sequences may specify that Labeled CBOR
Sequence labels can occur within a CBOR Sequence, possibly even to
switch to data items following in the sequence that are of a different type.If the CBOR-Sequence-based protocol does not define the semantics for
or at least tolerate embedded labels, care must be taken when
concatenating Labeled CBOR Sequences to remove the label from all but
the first part.If only one item is ever expected in the file, the use of the Labeled CBOR Sequence may present an implementation hurdle to programs that previously just read a single data item and used it.Are there tags at the start?If the Protocol expects to use other tags at its top level, then the
use of the CBOR Tag Wrapped format may be easy to explain at the same
place in the protocol description.CBOR Tags for CoAP Content Formats defines the concept of a Content-Format,
which is a short, 16-bit unsigned integer that identifies a specific
content type (media type plus (optionally) parameters), optionally
together with a content coding (see ).
Outside of a transfer protocol that indicates the Content-Format for a representation, it may be necessary to identify the Content-Format of the representation when it is stored in a file, in firmware, or when debugging.This specification allocates CBOR tag numbers 1668546817 (0x63740101) to
1668612095 (0x6374FFFF) for the tagging of representations of specific
content formats.Using tags from this range, a byte string that is to be interpreted as
a representation of Content-Format number ct, with ct < 65025
(255*255), can be identified by enclosing it in a tag with tag number
TN(ct) where:
(where +, *, / and % stand for integer addition,
multiplication, division, and remainder as in the programming language C .)Exceptionally, when used immediately as tag content of one of the tags
55799, 55800, or 55801, the tag content is as follows:
Tag 55799 ():
One of:
The CBOR data item within the representation (without byte-string
wrapping). This only works for Content-Formats that are represented
by a single CBOR data item in identity content-coding.
The data items in the CBOR sequence within the representation, without byte string
wrapping, but wrapped in a CBOR array.
This works for Content-Formats that are represented
by a CBOR sequence in identity content-coding.
Tags 55800 () or 55801 ():
the byte string 'BOR', signifying that the representation of the
given content-format follows in the file, in the way defined for these tags.
Content-Format Tag ExamplesThe "CoAP Content-Formats" registry defines content formats that can be used as examples:
As discussed in , Content-Format ID 112 represents the application/senml+cbor media type (no parameters).
The corresponding tag number is TN(112) = 1668546929.
The following CDDL snippet can be used to identify
application/senml+cbor representations:
senml-cbor = #6.1668546929(bstr)
Note that a byte string is used as the type of the tag content because a
media type representation in general can be any byte string.
Content-Format ID 272 represents the
application/missing-blocks+cbor-seq media type, which is a CBOR sequence
.
The corresponding tag number is TN(272) = 1668547090.
The following CDDL snippet can be used to identify
application/missing-blocks+cbor-seq representations as embedded in a
CBOR byte string:
missing-blocks = #6.1668547090(bstr)
Example from OpenswanThe Openswan IPsec project has a daemon ("pluto") and two control programs ("addconn" and "whack"). They communicate via a Unix-domain socket, over which a C-structure containing pointers to strings is serialized using a bespoke mechanism.
This is normally not a problem as the structure is compiled by the same compiler; but when there are upgrades, it is possible for the daemon and the control programs to get out of sync by the bespoke serialization.
As a result, there are extra compensations to deal with shutting the daemon down.
During testing, it is sometimes the case that upgrades are backed out.In addition, when doing unit testing, the easiest way to load policy is to use the
normal policy-reading process, but that is not normally loaded in the daemon.
Instead, the IPC that is normally sent across the wire is compiled, serialized, and placed in a file.
The above magic number is included in the file and on the IPC in order to distinguish the "shutdown" command CBOR operation.In order to reduce the problems due to serialization, the serialization is being changed to CBOR. Additionally, this change allows the IPC to be described by CDDL and any implementation language to be used that can encode CBOR.
IANA has allocated the tag 1330664270 or 0x4f_50_53_4e for this purpose.
As a result, each file and each IPC is prefixed with a CBOR Sequence tag.In diagnostic notation:
55800(1330664270(h'424F52'))
Or in hex:
d9 d9f8 # tag(55800)
da 4f50534e # tag(1330664270)
43 # bytes(3)
424f52 # "BOR"
Using CBOR Labels for Non-CBOR DataThe CBOR-Labeled Non-CBOR data method is appropriate for adding a magic number to a
Non-CBOR data format, particularly one that can be described by a
Content-Format tag ().This method prepends a CBOR data item to the Non-CBOR data; this data
item is called the "header" and, similar to the Labeled
CBOR-Sequence label, consists of two nested tags around a constant
byte string for a total of 12 bytes.
The outer tag is the CBOR-Labeled Non-CBOR Data tag, 55801.
The inner tag is a CBOR tag from the First Come First Served space that uniquely identifies the CBOR Protocol.
As with CBOR Tag Wrapped, the use of a 4-byte tag is encouraged that encodes without zero bytes.
The tag content is a 3-byte CBOR byte string containing
0x42_4F_52 ('BOR' in diagnostic notation).
The outer tag in the label identifies the file as being prefixed by a Non-CBOR
data label and does so with all
the desirable properties explained in .
Specifically, it does not appear to conflict with any known file types, and it is not valid Unicode in any Unicode encoding.The inner tag in the label identifies which Non-CBOR Protocol is used.The inner tag content is a constant byte string that is represented as
0x43_42_4f_52, the ASCII characters "CBOR",
which is the CBOR-encoded data item for the 3-byte string
0x42_4f_52 ('BOR' in diagnostic notation).The actual Non-CBOR Protocol data then follow directly appended to the
CBOR representation of the header.
This allows the application to trivially remove the header item with
the two nested tags and the byte string.As with the Labeled CBOR Sequence {#sequences}, this choice of the tag
content places the ASCII characters "CBOR" prominently into the
header.Content-Format Tag ExamplesThe "CoAP Content-Formats" registry defines content formats that can be used as examples:
Content-Format ID 432 represents the application/td+json media type (no
parameters).The corresponding tag number is TN(432) = 1668547250.
The following CDDL snippet can be used to identify a CBOR-Labeled Non-CBOR data for application/td+json representations:
td-json-header = #6.55801(#6.1668547250('BOR'))
Content-Format 11050 represents the application/json media type in
deflate content-coding.
The corresponding tag number is TN(11050) = 1668557910.
The following CDDL snippet can be used to identify a CBOR-Labeled Non-CBOR data for application/json representations compressed in deflate content-coding:
json-deflate-header = #6.55801(#6.1668557910('BOR'))
AcknowledgementsThe CBOR WG brainstormed this protocol on January 20, 2021 via a number of productive email exchanges on the mailing list.Contributorsjeffpc@josefsipek.netAuthors' AddressesSandelman Software Worksmcr+ietf@sandelman.caUniversität Bremen TZIPostfach 330440D-28359 BremenGermany+49-421-218-63921cabo@tzi.org