Compare commits

...

124 Commits

Author SHA1 Message Date
Faye Amacker
6f932b0868
Merge pull request #650 from fxamacker/fxamacker/update-stream-mode-with-v2.8
Some checks failed
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.24, macos-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.20, macos-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.20, ubuntu-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.20, windows-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.21, macos-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.21, ubuntu-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.21, windows-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.22, macos-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.22, ubuntu-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.22, windows-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.23, macos-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.23, ubuntu-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.23, windows-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.24, ubuntu-latest) (push) Has been cancelled
ci / test ${{matrix.os}} go-${{ matrix.go-version }} (1.24, windows-latest) (push) Has been cancelled
CodeQL / Analyze (go) (push) Has been cancelled
govulncheck / Check (push) Has been cancelled
Sync feature/stream-mode branch with v2.8.0
2025-04-02 14:40:37 -05:00
Faye Amacker
893e81ba89
Merge pull request #640 from fxamacker/fxamacker/update-stream-mode-with-v2.7
Sync feature/stream-mode branch with v2.7.0
2025-04-02 14:39:26 -05:00
Faye Amacker
8bb2c1a19c Merge tag 'v2.8.0' into fxamacker/update-stream-mode-with-v2.8 2025-04-01 11:59:20 -05:00
Faye Amacker
cbe7442e9e
Update docs for cbor v2.8.0 (#649)
Some checks failed
CodeQL / Analyze (go) (push) Has been cancelled
govulncheck / Check (push) Has been cancelled
Update README.md:
- Update "Status" section.
- Replace "struct tags" with "struct tag options" when referring to options.

Update doc.go:
- Mention omitzero option (for struct tags) in more places. Also mention Diagnose, DiagnoseFirst, UnmarshalFirst, etc. added in prior releases.
- Add example using UnmarshalFirst to show support for trailing bytes (extraneous data) typically found in CBOR Sequences (RFC 8742).
-  Add example using DiagnoseFirst to show support for representing binary CBOR data item in Extended Diagnostic Notation (human-readable text inspired by JSON).
- Replace "struct tags" with "struct tag options" when referring to options.

Update example_test.go
- Replace "struct tags" with "struct tag options" when referring to options.
2025-03-30 13:25:59 -05:00
Faye Amacker
c9f772d6d1
Merge pull request #647 from fxamacker/fxamacker/optimize-unmarshalcbor-for-cbor-builtin-structs
Optimize internal calls to UnmarshalCBOR() for ByteString, RawTag, SimpleValue
2025-03-28 09:27:52 -05:00
Faye Amacker
64fbdbc70d Optimize calls to UnmarshalCBOR() for RawTag, etc.
Currently, unreleased changes in PR #636 and #645 cause the
input data to be checked twice when UnmarshalCBOR() is
called internally by Unmarshal() for:
- ByteString
- RawTag
- SimpleValue

UnmarshalCBOR() checks input data because it can be called by
user apps providing bad data. However, the codec already checks
input data before internally calling UnmarshalCBOR() so the
2nd check is redundant.

This commit avoids redundant check on the input data by having
Unmarshal() call the private unmarshalCBOR() if implemented
by ByteString, RawTag, SimpleValue, etc.:
- Internally, the codec calls the private unmarshalCBOR() to
  avoid the redundant check on input data.
- Externally, UnmarshalCBOR() is available as a wrapper that
  checks input data before calling the private unmarshalCBOR().

UnmarshalCBOR() for ByteString, RawTag, and SimpleValue are marked
as deprecated and Unmarshal() should be used instead.
2025-03-27 19:20:08 -05:00
Faye Amacker
fe81c11acb
Merge pull request #645 from fxamacker/fxamacker/port-pr-636-to-master-branch
Port updated error handling in RawTag.UnmarshalCBOR(), etc. to match cbor.Unmarshal()
2025-03-26 22:37:15 -05:00
Faye Amacker
d81ecdda4b Check CBOR well-formedness in ByteString.UnmarshalCBOR
When ByteString.UnmarshalCBOR() is called by codec
(normal case), the codec will first check if data is well-formed
before calling ByteString.UnmarshalCBOR(data).

However, it can also be called by user app (not intended use)
and user apps might not check if data is well-formed.  In
such cases, this function can panic if given malformed data.

This commit updates ByteString.UnmarshalCBOR() to check for
well-formedness inside the function, so it behaves the same
whether it is called by codec internally or by user app.

Unfortunately, this approach means the same data is checked twice
for the normal case of the codec using
Unmarshal(data, *ByteString).

This can be revisited and maybe optimized in the future.
2025-03-26 16:27:15 -05:00
Faye Amacker
4f3c6a5886 Check CBOR well-formedness in SimpleValue.UnmarshalCBOR
When SimpleValue.UnmarshalCBOR() is called by codec
(normal case), the codec will first check if data is well-formed
before calling SimpleValue.UnmarshalCBOR(data).

However, it can also be called by user app (not intended use)
and user apps might not check if data is well-formed.  In
such cases, this function can panic if given malformed data.

This commit updates SimpleValue.UnmarshalCBOR() to check for
well-formedness inside the function, so it behaves the same
whether it is called by codec internally or by user app.

Unfortunately, this approach means the same data is checked twice
for the normal case of the codec using
Unmarshal(data, *SimpleValue).

This can be revisited and maybe optimized in the future.
2025-03-26 16:27:05 -05:00
Faye Amacker
621c6665ca Check CBOR well-formedness in RawTag.UnmarshalCBOR
When RawTag.UnmarshalCBOR() is called by codec (normal case),
the codec will first check if data is well-formed before
calling RawTag.UnmarshalCBOR(data).

However, it can also be called by user app (not intended use)
and user apps might not check if data is well-formed.  In
such cases, this function can panic if given malformed data.

This commit updates RawTag.UnmarshalCBOR() to check for
well-formedness inside the function, so it behaves the same
whether it is called by codec internally or by user app.

Unfortunately, this approach means the same data is checked twice
for the normal case of the codec using Unmarshal(data, *RawTag).
This can be revisited and maybe optimized in the future.
2025-03-26 16:26:53 -05:00
Faye Amacker
d34c8ec592
Merge pull request #644 from liggitt/omitzero
Add omitzero support
2025-03-26 15:21:05 -05:00
Jordan Liggitt
69da12b0b4
Adjust OmitZero tests to zero behavior
Signed-off-by: Jordan Liggitt <liggitt@google.com>
2025-03-26 09:55:55 -04:00
Jordan Liggitt
101dea75b0
Copy OmitEmpty tests to OmitZero tests
Signed-off-by: Jordan Liggitt <liggitt@google.com>
2025-03-26 09:55:54 -04:00
Jordan Liggitt
167dc75c19
Add omitzero support
Signed-off-by: Jordan Liggitt <liggitt@google.com>
2025-03-26 09:55:53 -04:00
Faye Amacker
a2a5c5ae80
Merge pull request #642 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.13
Bump github/codeql-action from 3.28.12 to 3.28.13
2025-03-25 08:14:42 -05:00
dependabot[bot]
d395852ddb
Bump github/codeql-action from 3.28.12 to 3.28.13
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.12 to 3.28.13.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](5f8171a638...1b549b9259)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-25 09:26:15 +00:00
Faye Amacker
166983ea24
Merge pull request #641 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.12
Bump github/codeql-action from 3.28.11 to 3.28.12
2025-03-22 10:21:51 -05:00
Faye Amacker
fcafe06327
Merge pull request #639 from fxamacker/dependabot/github_actions/actions/setup-go-5.4.0
Bump actions/setup-go from 5.3.0 to 5.4.0
2025-03-22 10:21:29 -05:00
Faye Amacker
7c72f96160
Merge pull request #637 from sschulz-t/fix/tag-doc
fix typo in example (use em)
2025-03-22 09:58:16 -05:00
Faye Amacker
00478930d2 Restore MaxMapPairs limit for feature/stream-mode
This is needed for backward compatibility with
a project that sets this limit to math.MaxInt64.
2025-03-21 16:54:04 -05:00
dependabot[bot]
f30c336bbe
Bump github/codeql-action from 3.28.11 to 3.28.12
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.11 to 3.28.12.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](6bb031afdd...5f8171a638)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-20 09:51:07 +00:00
Faye Amacker
379167c29d Lint 2025-03-19 10:57:30 -05:00
Faye Amacker
62b894b173 Merge tag 'v2.7.0' into fxamacker/update-stream-mode-with-master 2025-03-19 10:17:47 -05:00
dependabot[bot]
7f0fef7083
Bump actions/setup-go from 5.3.0 to 5.4.0
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.3.0 to 5.4.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](f111f3307d...0aaccfd150)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-19 09:32:10 +00:00
Simon Schulz
337ac607b5 fix typo in example 2025-03-18 12:43:25 +01:00
Faye Amacker
b1949d2397
Merge pull request #633 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.11
Bump github/codeql-action from 3.28.10 to 3.28.11
2025-03-12 23:11:05 -05:00
dependabot[bot]
cf5fbfb812
Bump github/codeql-action from 3.28.10 to 3.28.11
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.10 to 3.28.11.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](b56ba49b26...6bb031afdd)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 09:42:16 +00:00
Faye Amacker
5ab6c9281a
Merge pull request #631 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.10
Bump github/codeql-action from 3.28.9 to 3.28.10
2025-03-02 16:38:28 -06:00
dependabot[bot]
9cd84f26d1
Bump github/codeql-action from 3.28.9 to 3.28.10
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.9 to 3.28.10.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](9e8d0789d4...b56ba49b26)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-24 09:50:32 +00:00
Faye Amacker
9357f00454
Merge pull request #630 from fxamacker/fxamacker/add-missing-copyright-to-a-few-files
Add missing copyright notice to a few files
2025-02-23 16:59:28 -06:00
Faye Amacker
728e5ac128
Add missing copyright notice to json_test.go 2025-02-23 16:57:39 -06:00
Faye Amacker
fcb7848012
Add missing copyright notice to tag_test.go 2025-02-23 16:40:26 -06:00
Faye Amacker
d793074e33
Add missing copyright notice to tag.go 2025-02-23 16:39:49 -06:00
Faye Amacker
94619f8cec
Add missing copyright notice to simplevalue.go 2025-02-23 16:33:52 -06:00
Faye Amacker
0263af4182
Merge pull request #629 from fxamacker/fxamacker/replace-reflect-ptrto
Replace `reflect.PtrTo` with `reflect.PointerTo`
2025-02-22 14:34:41 -06:00
Faye Amacker
c26c5a219e
Merge pull request #628 from fxamacker/fxamacker/replace-reflect-ptr
Replace reflect.Ptr with reflect.Pointer
2025-02-22 14:33:57 -06:00
Faye Amacker
34c6f2b442 Replace reflect.PtrTo with reflect.PointerTo 2025-02-22 14:25:35 -06:00
Faye Amacker
2e3b625a2f
Merge pull request #627 from fxamacker/fxamacker/replace-empty-interface
Replace interface{} with any
2025-02-22 14:21:49 -06:00
Faye Amacker
50b362d72f Replace reflect.Ptr with reflect.Pointer
This commit replaces reflect.Ptr with reflect.Pointer,
so go1.18 or newer is required to to build.
2025-02-22 14:20:19 -06:00
Faye Amacker
9fa2077793 Replace interface{} with any
This commit replaces every instance of interface{} with any,
so go1.18 or newer is required to to build.
2025-02-22 14:05:53 -06:00
Faye Amacker
57985adf1d
Merge pull request #626 from fxamacker/fxamacker/update-to-require-go1.20
go: update go.mod and ci.yml to require go1.20 or newer (was go1.17)
2025-02-22 13:51:51 -06:00
Faye Amacker
40e05c4d19
Remove build tag in encode_map.go 2025-02-22 12:39:23 -06:00
Faye Amacker
f406e90b38
Delete encode_map_go117.go 2025-02-22 12:33:38 -06:00
Faye Amacker
eccd7b1880
go.mod: bump go from 1.17 to 1.20 2025-02-22 12:24:02 -06:00
Faye Amacker
0eabc5f7af
.github: update ci.yml to test go1.20-1.24
Update ci.yml to test latest 5 versions of go.
2025-02-22 12:12:26 -06:00
Faye Amacker
2b74e0134d
Merge pull request #621 from fxamacker/fxamacker/require-97-percent-code-coverage
.github: Require 97% or more code coverage
2025-02-16 22:34:10 -06:00
Faye Amacker
d7cb68ea8f
Merge pull request #620 from fxamacker/fxamacker/bump-govulncheck-to-1.1.4
.github: Bump govulncheck from 1.1.3 to 1.1.4
2025-02-16 22:33:08 -06:00
Faye Amacker
9a1f6311b0
README: Update URL to code coverage badge
This commit updates the code coverage URL to

https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A597%25/badge.svg

This badge.svg will show "passing" if code coverage
is at least 97%.
2025-02-16 20:29:07 -06:00
Faye Amacker
c15799871f
.github: Require 97% or more code coverage
Currently, "go test -short -cover" reports 97.6% coverage.

This commit updates ci-go-cover.yml to return error if
"go test -short -cover" reports less than 97% coverage.

Also bump go 1.23 to 1.24 for running coverage test.
2025-02-16 20:15:02 -06:00
Faye Amacker
b7d18f2be1
.github: Bump govulncheck from 1.1.3 to 1.1.4 2025-02-16 19:36:36 -06:00
Faye Amacker
525ad91862
Merge pull request #619 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.9
Bump github/codeql-action from 3.28.8 to 3.28.9
2025-02-10 08:27:09 -06:00
dependabot[bot]
8b7db35d04
Bump github/codeql-action from 3.28.8 to 3.28.9
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.8 to 3.28.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](dd746615b3...9e8d0789d4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-10 09:15:47 +00:00
Faye Amacker
b3f38d2530
Merge pull request #618 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.8
Bump github/codeql-action from 3.28.4 to 3.28.8
2025-02-01 21:24:41 -06:00
dependabot[bot]
8e50eaf1c9
Bump github/codeql-action from 3.28.4 to 3.28.8
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.4 to 3.28.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](ee117c905a...dd746615b3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-30 09:54:32 +00:00
Faye Amacker
7f356f9a8d
Merge pull request #614 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.4
Bump github/codeql-action from 3.28.1 to 3.28.4
2025-01-26 09:55:03 -06:00
Faye Amacker
cf2cb89c58
Merge pull request #610 from fxamacker/dependabot/github_actions/actions/setup-go-5.3.0
Bump actions/setup-go from 5.2.0 to 5.3.0
2025-01-26 09:54:38 -06:00
dependabot[bot]
ced1b37795
Bump github/codeql-action from 3.28.1 to 3.28.4
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.1 to 3.28.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](b6a472f63d...ee117c905a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-24 09:24:38 +00:00
dependabot[bot]
82e1a540aa
Bump actions/setup-go from 5.2.0 to 5.3.0
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.2.0 to 5.3.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](3041bf56c9...f111f3307d)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-21 09:40:56 +00:00
Faye Amacker
dc281024cb
Merge pull request #609 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.1
Bump github/codeql-action from 3.28.0 to 3.28.1
2025-01-18 23:14:58 -06:00
dependabot[bot]
a16f7faa51
Bump github/codeql-action from 3.28.0 to 3.28.1
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.0 to 3.28.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](48ab28a6f5...b6a472f63d)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-13 09:27:43 +00:00
Faye Amacker
4c7fe52c5d
Merge pull request #608 from fxamacker/dependabot/github_actions/github/codeql-action-3.28.0
Bump github/codeql-action from 3.27.9 to 3.28.0
2024-12-31 15:23:46 -06:00
dependabot[bot]
5c6cbc9801
Bump github/codeql-action from 3.27.9 to 3.28.0
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.9 to 3.28.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](df409f7d92...48ab28a6f5)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-23 09:29:25 +00:00
Faye Amacker
2d0787bd8a
Merge pull request #607 from fxamacker/dependabot/github_actions/github/codeql-action-3.27.9
Bump github/codeql-action from 3.27.7 to 3.27.9
2024-12-15 16:13:12 -06:00
dependabot[bot]
d68944fb08
Bump github/codeql-action from 3.27.7 to 3.27.9
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.7 to 3.27.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](babb554ede...df409f7d92)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-13 09:16:15 +00:00
Faye Amacker
8baf0c2fd9
Merge pull request #606 from fxamacker/dependabot/github_actions/actions/setup-go-5.2.0
Bump actions/setup-go from 5.1.0 to 5.2.0
2024-12-11 22:34:58 -06:00
Faye Amacker
36bbe36074
Merge pull request #605 from fxamacker/dependabot/github_actions/github/codeql-action-3.27.7
Bump github/codeql-action from 3.27.6 to 3.27.7
2024-12-11 22:34:32 -06:00
dependabot[bot]
7391fe8524
Bump actions/setup-go from 5.1.0 to 5.2.0
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.1.0 to 5.2.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](41dfa10bad...3041bf56c9)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-11 09:45:57 +00:00
dependabot[bot]
d4065cf3a1
Bump github/codeql-action from 3.27.6 to 3.27.7
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.6 to 3.27.7.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](aa57810251...babb554ede)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-11 09:45:53 +00:00
Faye Amacker
7398b0dd82
Merge pull request #604 from fxamacker/dependabot/github_actions/github/codeql-action-3.27.6
Bump github/codeql-action from 3.27.4 to 3.27.6
2024-12-08 16:50:24 -06:00
Faye Amacker
2b757e6e04
Update README.md
Add IBM to list of users.
2024-12-08 16:47:41 -06:00
dependabot[bot]
64df230cbe
Bump github/codeql-action from 3.27.4 to 3.27.6
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.4 to 3.27.6.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](ea9e4e3799...aa57810251)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-04 09:52:27 +00:00
Faye Amacker
62b2e360ba
Merge pull request #602 from fxamacker/dependabot/github_actions/github/codeql-action-3.27.4
Bump github/codeql-action from 3.27.0 to 3.27.4
2024-11-17 11:30:39 -06:00
dependabot[bot]
d38afd46ec
Bump github/codeql-action from 3.27.0 to 3.27.4
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.0 to 3.27.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](662472033e...ea9e4e3799)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-15 09:07:54 +00:00
Faye Amacker
696cd83ac2
Merge pull request #598 from fxamacker/dependabot/github_actions/actions/setup-go-5.1.0
Bump actions/setup-go from 5.0.2 to 5.1.0
2024-11-09 06:59:51 -06:00
dependabot[bot]
3ce8dda993
Bump actions/setup-go from 5.0.2 to 5.1.0
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.0.2 to 5.1.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](0a12ed9d6a...41dfa10bad)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-05 13:29:34 +00:00
Faye Amacker
a2f5a4cdc4
Merge pull request #597 from fxamacker/dependabot/github_actions/actions/checkout-4.2.2
Bump actions/checkout from 4.2.1 to 4.2.2
2024-11-05 07:28:46 -06:00
Faye Amacker
047288a2ae
Merge pull request #596 from fxamacker/dependabot/github_actions/github/codeql-action-3.27.0
Bump github/codeql-action from 3.26.13 to 3.27.0
2024-11-03 16:15:04 -06:00
dependabot[bot]
d0435e47f1
Bump actions/checkout from 4.2.1 to 4.2.2
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.1 to 4.2.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](eef61447b9...11bd71901b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-24 09:30:43 +00:00
dependabot[bot]
e268fee497
Bump github/codeql-action from 3.26.13 to 3.27.0
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.13 to 3.27.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](f779452ac5...662472033e)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-23 09:25:27 +00:00
Faye Amacker
b2962623de
Merge pull request #595 from fxamacker/dependabot/github_actions/github/codeql-action-3.26.13
Bump github/codeql-action from 3.26.12 to 3.26.13
2024-10-20 13:28:22 -05:00
dependabot[bot]
ae3bcc3e77
Bump github/codeql-action from 3.26.12 to 3.26.13
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.12 to 3.26.13.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](c36620d31a...f779452ac5)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-15 09:19:14 +00:00
Faye Amacker
10f5fdc44b
Merge pull request #593 from fxamacker/dependabot/github_actions/actions/checkout-4.2.1
Bump actions/checkout from 4.2.0 to 4.2.1
2024-10-12 12:23:12 -05:00
Faye Amacker
b7a1d226a0
.github: run govulncheck daily (was weekly) 2024-10-12 12:15:37 -05:00
Faye Amacker
057222d3ed
.github: run govulncheck on all branches and paths 2024-10-12 12:07:40 -05:00
Faye Amacker
8b9125ba24
.github: update push branches in ci.yml 2024-10-12 12:02:24 -05:00
Faye Amacker
b43cf6f0ca
Merge pull request #594 from fxamacker/dependabot/github_actions/github/codeql-action-3.26.12
Bump github/codeql-action from 3.26.11 to 3.26.12
2024-10-09 18:39:37 -05:00
dependabot[bot]
f4132d4af3
Bump github/codeql-action from 3.26.11 to 3.26.12
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.11 to 3.26.12.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](6db8d6351f...c36620d31a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-08 09:19:23 +00:00
dependabot[bot]
c0c0026367
Bump actions/checkout from 4.2.0 to 4.2.1
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.0 to 4.2.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](d632683dd7...eef61447b9)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-08 09:19:14 +00:00
Faye Amacker
8c81f98a7f
Merge pull request #592 from fxamacker/fxamacker/update-image-url
README: fix broken link to svg file
2024-10-06 00:18:37 -05:00
Faye Amacker
0816831c34
README: update link to image
Update link to image in README.md.
2024-10-06 00:09:24 -05:00
Faye Amacker
3dd6572da2
README: update to clarify CBOR benchmark comparison and resolve nits (#591)
Update README.md.
2024-10-05 23:17:49 -05:00
Faye Amacker
b8f4ad5643
Merge pull request #590 from fxamacker/dependabot/github_actions/github/codeql-action-3.26.11
Bump github/codeql-action from 3.26.9 to 3.26.11
2024-10-05 16:04:27 -05:00
Faye Amacker
a9d2bc92fe
.github: update codeql-analysis.yml
Run on pull request for all branches.
Run on push for all branches.
Run daily at 12:30 UTC (7:30 AM Central).
2024-10-05 15:54:48 -05:00
dependabot[bot]
c28eb653a1
Bump github/codeql-action from 3.26.9 to 3.26.11
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.9 to 3.26.11.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](461ef6c76d...6db8d6351f)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-04 09:15:13 +00:00
Faye Amacker
a5930df35b
Merge pull request #588 from fxamacker/fxamacker/update-readme-cbor-benchmarks
README: update benchmark comparisons, etc.
2024-09-29 22:37:15 -05:00
Faye Amacker
20d7fbce6c
README: update benchmark comparisons, etc.
Update benchmarks with fxamacker/cbor v2.7.0 for
rejecting malformed CBOR data while decoding.

While at it, also improved formatting of the first page.
2024-09-29 21:30:53 -05:00
dependabot[bot]
7c4b10c00a
Bump actions/checkout from 4.1.7 to 4.2.0 (#587)
* Bump actions/checkout from 4.1.7 to 4.2.0

* Add concurrency group to ci-go-cover.yml

* Add concurrency group to ci.yml

* Add concurrency group to govulncheck.yml  
  While at it, also add cron schedule.

* Add concurrency group to safer-golangci-lint.yml  
  While at it, bump golangci-lint to 1.59.1.

* Bump go to 1.23 in govulncheck.yml

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Faye Amacker <33205765+fxamacker@users.noreply.github.com>
2024-09-29 13:44:48 -05:00
Faye Amacker
46c3919161
Merge pull request #586 from fxamacker/dependabot/github_actions/github/codeql-action-3.26.9
Bump github/codeql-action from 3.26.8 to 3.26.9
2024-09-25 20:56:30 -05:00
Faye Amacker
1af2a8c757
Add comments to codeql-analysis.yml
Add comments for cron and make it run on Sunday mornings.
2024-09-25 20:54:23 -05:00
dependabot[bot]
28ecbc4ad7
Bump github/codeql-action from 3.26.8 to 3.26.9
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.8 to 3.26.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](294a9d9291...461ef6c76d)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-25 09:34:29 +00:00
Faye Amacker
3c21ddd603
Merge pull request #585 from fxamacker/dependabot/github_actions/github/codeql-action-3.26.8
Bump github/codeql-action from 3.26.6 to 3.26.8
2024-09-21 17:56:32 -05:00
Faye Amacker
b28c1f17a4
Add concurrency group to codeql-analysis.yml 2024-09-21 17:53:17 -05:00
dependabot[bot]
d0ba62f16c
Bump github/codeql-action from 3.26.6 to 3.26.8
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.6 to 3.26.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](4dd16135b6...294a9d9291)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-20 09:44:45 +00:00
Faye Amacker
9d631afeec
Merge pull request #582 from fxamacker/fxamacker/mention-tinygo-feature-branch-in-master-branch
Mention new TinyGo feature branch in README
2024-09-08 13:50:19 -05:00
Faye Amacker
c67eb1449b
Mention new Tinygo feature branch in README
Update Quick Start section of README to mention new branch:
- feature/cbor-tinygo-beta

It can be compiled using tinygo v0.33+ and go1.20+.
2024-09-08 11:59:00 -05:00
Faye Amacker
c0c9f774e4
Merge pull request #392 from fxamacker/fxamacker/add-streamencoder-close
[stream-mode branch] Add StreamEncoder.Close() to recycle buffer
2023-02-28 11:37:56 -06:00
Faye Amacker
ec12818892 Add StreamEncoder.Close() to recycle buffer 2023-02-28 11:13:08 -06:00
Faye Amacker
01590f19b2
Merge pull request #349 from fxamacker/fxamacker/support-tinygo
[stream-mode branch] Enable tinygo 0.23 to compile and use CBOR StreamEncoder and StreamDecoder
2022-06-05 21:26:38 -05:00
Faye Amacker
a5817116ab Support tinygo for StreamEncoder and StreamDecoder
Replaced reflect.MakeMapWithSize() with reflect.MakeMap()
to allow compiling with tinygo 0.23 (latest version).

Only StreamEncoder and StreamDecoder can be used with
tinygo.  Other features may encounter stubs in tinygo
that panics due to unimplmented Go features.

For more details, see
https://github.com/fxamacker/cbor/issues/295#issuecomment-1140339662
2022-05-28 18:25:19 -05:00
Faye Amacker
ad2eae6330 Merge branch 'master' into feature/stream-mode 2022-05-15 13:34:30 -05:00
Faye Amacker
5b61085419
Merge pull request #343 from fxamacker/fxamacker/merge-master-into-stream-mode
merge master into feature/stream-mode branch
2022-05-15 12:31:40 -05:00
Faye Amacker
cc6250ef15 Merge branch 'master' into fxamacker/merge-master-into-stream-mode 2022-05-15 12:10:03 -05:00
Faye Amacker
12f5cb4b5e
Add support for streaming CBOR map (#327)
- Add support for stream encoding CBOR map
- Add support for stream decoding CBOR map
2022-03-13 20:10:55 -05:00
Faye Amacker
1fafba071f
Support CBOR bignum in NextSize() (#325)
Add requested feature in stream-mode branch.
2022-03-08 12:08:23 -06:00
Faye Amacker
8cc0fef5e5
Add decoding option to allow invalid UTF-8 (#323)
Add DecOptions.UTF8 to specify how to decode invalid UTF-8:
- UTF8RejectInvalid (default) returns SemanticError on invalid UTF-8
- UTF8DecodeInvalid decodes invalid UTF-8

Since CBOR text containing invalid UTF-8 isn't valid, decoder by default
rejects it and returns error.

This option is backward compatible by default.

See RFC 8949 Section 5.3 (Validity of Items) for more info.
2022-03-03 13:16:04 -06:00
Faye Amacker
a68253a1c8 Merge branch 'master' into feature/stream-mode 2022-01-31 12:02:38 -06:00
Faye Amacker
3b6925d507
Add NextSize() to stream decoder (#322)
NextSize() returns next CBOR data size for four CBOR types:
- byte string: length (in bytes) of byte string
- text string: length (in bytes) of text string
- array: number of array elements
- map: number of key/value pairs

Error is returned for indef length data and other CBOR types.
2022-01-31 11:55:32 -06:00
Faye Amacker
5d5d7c3edd
Merge pull request #300 from fxamacker/master
- Feature: Allow decoding to struct field of interface type (commit 4a03f1c)
- Updates to README.md and other docs
- Updates to CI

Co-authored-by: Montgomery Edwards⁴⁴⁸ <57072051+x448@users.noreply.github.com>
Co-authored-by: Stefan Tatschner <stefan@rumpelsepp.org>
Co-authored-by: Ackermann Yuriy <1636116+herrjemand@users.noreply.github.com>
2021-10-29 11:21:00 -05:00
Faye Amacker
3d6d5d1de2
Add NumBytesDecoded() to StreamDecoder (#299) 2021-09-27 18:51:16 -05:00
Faye Amacker
c3f3c69e7b
Expose fields of WrongTypeError and other improvements (#271)
Expose fields of WrongTypeError so caller can get more details.

Replace string concat with fmt.Sprintf to construct error message. 
(Thanks @turbolent for suggestion!)

Reuse default mode NewStreamDecoder and NewByteStreamDecoder 
in custom mode NewStreamDecoder and NewByteStreamDecoder. 
(Thanks @turbolent for suggestion!)
2021-05-10 14:28:46 -05:00
Faye Amacker
6eb63b995e
Add stream decoding (#270)
StreamDecoder provides low-level API for sequential decoding.

It can be used to avoid creating large collection objects.

Users of StreamDecoder should be familiar with CBOR data models.
When in doubt, please use Decoder instead.
2021-05-10 10:46:07 -05:00
Faye Amacker
7ceed0a5d0 Increase range of max limits
- Increase MaxArrayElements from 2147483647 to math.MaxInt64 on 64-bit
arch

- Increase MaxMapPairs from 2147483647 to math.MaxInt64 on 64-bit arch

- Increase MaxNestedLevels from 256 to math.MaxInt16 on all arch

Default max limits are unchanged.
2021-04-27 20:10:03 -05:00
Faye Amacker
6fd2b2e8c6
Add low-level API to StreamEncoder (#267)
* Modify stream encoding API to return error

* Add low-level API to StreamEncoder
2021-04-21 22:02:33 -05:00
Faye Amacker
980a4ddd8f
Add stream encoding for CBOR array and tag (#266)
StreamEncoder provides low-level API for sequential encoding.

It can be used to avoid creating large collection objects.

Users of StreamEncoder should be familiar with CBOR data models.
When in doubt, please use Encoder instead.

Users should use Valid() to check whether generated CBOR data
is complete and well-formed.
2021-04-14 22:51:55 -05:00
33 changed files with 6706 additions and 985 deletions

View File

@ -14,7 +14,7 @@
# 1. Change workflow name from "cover 100%" to "cover ≥92.5%". Script will automatically use 92.5%.
# 2. Update README.md to use the new path to badge.svg because the path includes the workflow name.
name: cover ≥96%
name: cover ≥97%
# Remove default permissions.
permissions: {}
@ -25,6 +25,10 @@ on:
push:
branches: [main, master]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
# Verify minimum coverage is reached using `go test -short -cover` on latest-ubuntu with default version of Go.
@ -36,11 +40,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Go
uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0
with:
go-version: 1.21
go-version: 1.24
check-latest: true
- name: Install x448/float16
run: go get github.com/x448/float16@v0.8.4

View File

@ -11,11 +11,16 @@ on:
pull_request:
push:
branches:
- 'main'
- 'master'
- 'release*'
- 'feature/stream-mode'
tags:
- 'v*'
- 'release**' # Match both 'release-v2.3.4' and 'release/*'.
- 'feature/**'
- 'v**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
# Test on various OS with default Go version.
tests:
@ -26,16 +31,16 @@ jobs:
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
go-version: [1.17, 1.18, 1.19, '1.20', 1.21, 1.22, 1.23]
go-version: ['1.20', 1.21, 1.22, 1.23, 1.24]
steps:
- name: Install Go
uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Checkout code
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 1

View File

@ -5,12 +5,15 @@ permissions: {}
on:
push:
branches: [ master ]
# Run on push for all branches.
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
# Run on pull request for all branches.
schedule:
- cron: '30 5 * * 4'
- cron: '30 12 * * *' # Run daily at 12:30 UTC / 7:30 AM Central.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
analyze:
@ -28,18 +31,20 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@4dd16135b69a43b6c8efb853346f8437d92d3c93 # v3.26.6
uses: github/codeql-action/init@1b549b9259bda1cb5ddde3b41741a82a2d15a841 # v3.28.13
with:
languages: ${{ matrix.language }}
# Don't need setup-go because default version of Go is updated regularly in ubuntu-latest.
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@4dd16135b69a43b6c8efb853346f8437d92d3c93 # v3.26.6
uses: github/codeql-action/autobuild@1b549b9259bda1cb5ddde3b41741a82a2d15a841 # v3.28.13
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@4dd16135b69a43b6c8efb853346f8437d92d3c93 # v3.26.6
uses: github/codeql-action/analyze@1b549b9259bda1cb5ddde3b41741a82a2d15a841 # v3.28.13

View File

@ -10,20 +10,15 @@ permissions: {}
on:
workflow_dispatch:
pull_request:
paths:
- '**'
- '!**.md'
# Run govulncheck on PR for all branches and paths.
push:
paths:
- '**'
- '!**.md'
branches:
- 'main'
- 'master'
- 'release*'
- 'feature/stream-mode'
tags:
- 'v*'
# Run govulncheck on push for all branches and paths.
schedule:
- cron: '45 12 * * *' # Daily at 12:45 UTC / 7:45 AM Central
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
Check:
@ -33,15 +28,18 @@ jobs:
contents: read
steps:
- name: Checkout source
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 1
- name: Set up Go
uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
- name: Install Go and setup env
uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0
with:
go-version: 1.21.x
go-version: 1.23
check-latest: true
- name: Install latest from golang.org
run: go install golang.org/x/vuln/cmd/govulncheck@4ea4418106cea3bb2c9aa098527c924e9e1fbbb4 # v1.1.3
- name: Run govulncheck
- name: Install latest govulncheck
run: go install golang.org/x/vuln/cmd/govulncheck@d1f380186385b4f64e00313f31743df8e4b89a77 # v1.1.4
- name: Run govulncheck
run: govulncheck -show=traces ./...

View File

@ -17,13 +17,17 @@ on:
env:
GO_VERSION: '1.22'
GOLINTERS_VERSION: 1.56.2
GOLINTERS_VERSION: 1.59.1
GOLINTERS_ARCH: linux-amd64
GOLINTERS_TGZ_DGST: e1c313fb5fc85a33890fdee5dbb1777d1f5829c84d655a47a55688f3aad5e501
GOLINTERS_TGZ_DGST: c30696f1292cff8778a495400745f0f9c0406a3f38d8bb12cef48d599f6c7791
GOLINTERS_TIMEOUT: 15m
OPENSSL_DGST_CMD: openssl dgst -sha256 -r
CURL_CMD: curl --proto =https --tlsv1.2 --location --silent --show-error --fail
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
main:
name: Lint
@ -32,12 +36,12 @@ jobs:
contents: read
steps:
- name: Checkout source
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 1
- name: Setup Go
uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0
with:
go-version: ${{ env.GO_VERSION }}
check-latest: true

396
README.md
View File

@ -1,6 +1,4 @@
# CBOR Codec in Go
<!-- [![](https://github.com/fxamacker/images/raw/master/cbor/v2.5.0/fxamacker_cbor_banner.png)](#cbor-library-in-go) -->
<h1>CBOR Codec <a href="https://pkg.go.dev/github.com/fxamacker/cbor/v2"><img src="https://raw.githubusercontent.com/fxamacker/images/refs/heads/master/cbor/go-logo-blue.svg" alt="Go logo" style="height: 1em;" align="right"></a></h1>
[fxamacker/cbor](https://github.com/fxamacker/cbor) is a library for encoding and decoding [CBOR](https://www.rfc-editor.org/info/std94) and [CBOR Sequences](https://www.rfc-editor.org/rfc/rfc8742.html).
@ -8,23 +6,26 @@ CBOR is a [trusted alternative](https://www.rfc-editor.org/rfc/rfc8949.html#name
`fxamacker/cbor` is used in projects by Arm Ltd., Cisco, EdgeX&nbsp;Foundry, Flow Foundation, Fraunhofer&#8209;AISEC, Kubernetes, Let's&nbsp;Encrypt (ISRG), Linux&nbsp;Foundation, Microsoft, Mozilla, Oasis&nbsp;Protocol, Tailscale, Teleport, [etc](https://github.com/fxamacker/cbor#who-uses-fxamackercbor).
See [Quick&nbsp;Start](#quick-start) and [Releases](https://github.com/fxamacker/cbor/releases/). 🆕 `UnmarshalFirst` and `DiagnoseFirst` can decode CBOR Sequences. `cbor.MarshalToBuffer()` and `UserBufferEncMode` accepts user-specified buffer.
See [Quick&nbsp;Start](#quick-start) and [Releases](https://github.com/fxamacker/cbor/releases/). 🆕 `UnmarshalFirst` and `DiagnoseFirst` can decode CBOR Sequences. `MarshalToBuffer` and `UserBufferEncMode` accepts user-specified buffer.
## fxamacker/cbor
[![](https://github.com/fxamacker/cbor/workflows/ci/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3Aci)
[![](https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A596%25/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A596%25%22)
[![](https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A597%25/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A597%25%22)
[![CodeQL](https://github.com/fxamacker/cbor/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/fxamacker/cbor/actions/workflows/codeql-analysis.yml)
[![](https://img.shields.io/badge/fuzzing-passing-44c010)](#fuzzing-and-code-coverage)
[![Go Report Card](https://goreportcard.com/badge/github.com/fxamacker/cbor)](https://goreportcard.com/report/github.com/fxamacker/cbor)
[![](https://img.shields.io/ossf-scorecard/github.com/fxamacker/cbor?label=openssf%20scorecard)](https://github.com/fxamacker/cbor#fuzzing-and-code-coverage)
`fxamacker/cbor` is a CBOR codec in full conformance with [IETF STD&nbsp;94 (RFC&nbsp;8949)](https://www.rfc-editor.org/info/std94). It also supports CBOR Sequences ([RFC&nbsp;8742](https://www.rfc-editor.org/rfc/rfc8742.html)) and Extended Diagnostic Notation ([Appendix G of RFC&nbsp;8610](https://www.rfc-editor.org/rfc/rfc8610.html#appendix-G)).
Features include full support for CBOR tags, [Core Deterministic Encoding](https://www.rfc-editor.org/rfc/rfc8949.html#name-core-deterministic-encoding), duplicate map key detection, etc.
API is mostly same as `encoding/json`, plus interfaces that simplify concurrency and CBOR options.
Design balances trade-offs between security, speed, concurrency, encoded data size, usability, etc.
<details><summary>Highlights</summary><p/>
<details><summary> 🔎&nbsp; Highlights</summary><p/>
__🚀&nbsp; Speed__
@ -38,7 +39,7 @@ Codec passed multiple confidential security assessments in 2022. No vulnerabili
__🗜&nbsp; Data Size__
Struct tags (`toarray`, `keyasint`, `omitempty`) automatically reduce size of encoded structs. Encoding optionally shrinks float64→32→16 when values fit.
Struct tag options (`toarray`, `keyasint`, `omitempty`, `omitzero`) automatically reduce size of encoded structs. Encoding optionally shrinks float64→32→16 when values fit.
__:jigsaw:&nbsp; Usability__
@ -58,164 +59,201 @@ Features include CBOR [extension points](https://www.rfc-editor.org/rfc/rfc8949.
`fxamacker/cbor` has configurable limits, etc. that defend against malicious CBOR data.
By contrast, `encoding/gob` is [not designed to be hardened against adversarial inputs](https://pkg.go.dev/encoding/gob#hdr-Security).
Notably, `fxamacker/cbor` is fast at rejecting malformed CBOR data.
<details><summary>Example decoding with encoding/gob 💥 fatal error (out of memory)</summary><p/>
> [!NOTE]
> Benchmarks rejecting 10 bytes of malicious CBOR data decoding to `[]byte`:
>
> | Codec | Speed (ns/op) | Memory | Allocs |
> | :---- | ------------: | -----: | -----: |
> | fxamacker/cbor 2.7.0 | 47 ± 7% | 32 B/op | 2 allocs/op |
> | ugorji/go 1.2.12 | 5878187 ± 3% | 67111556 B/op | 13 allocs/op |
>
> Faster hardware (overclocked DDR4 or DDR5) can reduce speed difference.
>
> <details><summary> 🔎&nbsp; Benchmark details </summary><p/>
>
> Latest comparison for decoding CBOR data to Go `[]byte`:
> - Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
> - go1.22.7, linux/amd64, i5-13600K (DDR4-2933, disabled e-cores)
> - go test -bench=. -benchmem -count=20
>
> #### Prior comparisons
>
> | Codec | Speed (ns/op) | Memory | Allocs |
> | :---- | ------------: | -----: | -----: |
> | fxamacker/cbor 2.5.0-beta2 | 44.33 ± 2% | 32 B/op | 2 allocs/op |
> | fxamacker/cbor 0.1.0 - 2.4.0 | ~44.68 ± 6% | 32 B/op | 2 allocs/op |
> | ugorji/go 1.2.10 | 5524792.50 ± 3% | 67110491 B/op | 12 allocs/op |
> | ugorji/go 1.1.0 - 1.2.6 | 💥 runtime: | out of memory: | cannot allocate |
>
> - Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
> - go1.19.6, linux/amd64, i5-13600K (DDR4)
> - go test -bench=. -benchmem -count=20
>
> </details>
```Go
// Example of encoding/gob having "fatal error: runtime: out of memory"
// while decoding 181 bytes.
package main
import (
"bytes"
"encoding/gob"
"encoding/hex"
"fmt"
)
In contrast, some codecs can crash or use excessive resources while decoding bad data.
// Example data is from https://github.com/golang/go/issues/24446
// (shortened to 181 bytes).
const data = "4dffb503010102303001ff30000109010130010800010130010800010130" +
"01ffb80001014a01ffb60001014b01ff860001013001ff860001013001ff" +
"860001013001ff860001013001ffb80000001eff850401010e3030303030" +
"30303030303030303001ff3000010c0104000016ffb70201010830303030" +
"3030303001ff3000010c000030ffb6040405fcff00303030303030303030" +
"303030303030303030303030303030303030303030303030303030303030" +
"30"
> [!WARNING]
> Go's `encoding/gob` is [not designed to be hardened against adversarial inputs](https://pkg.go.dev/encoding/gob#hdr-Security).
>
> <details><summary> 🔎&nbsp; gob fatal error (out of memory) 💥 decoding 181 bytes</summary><p/>
>
> ```Go
> // Example of encoding/gob having "fatal error: runtime: out of memory"
> // while decoding 181 bytes (all Go versions as of Dec. 8, 2024).
> package main
> import (
> "bytes"
> "encoding/gob"
> "encoding/hex"
> "fmt"
> )
>
> // Example data is from https://github.com/golang/go/issues/24446
> // (shortened to 181 bytes).
> const data = "4dffb503010102303001ff30000109010130010800010130010800010130" +
> "01ffb80001014a01ffb60001014b01ff860001013001ff860001013001ff" +
> "860001013001ff860001013001ffb80000001eff850401010e3030303030" +
> "30303030303030303001ff3000010c0104000016ffb70201010830303030" +
> "3030303001ff3000010c000030ffb6040405fcff00303030303030303030" +
> "303030303030303030303030303030303030303030303030303030303030" +
> "30"
>
> type X struct {
> J *X
> K map[string]int
> }
>
> func main() {
> raw, _ := hex.DecodeString(data)
> decoder := gob.NewDecoder(bytes.NewReader(raw))
>
> var x X
> decoder.Decode(&x) // fatal error: runtime: out of memory
> fmt.Println("Decoding finished.")
> }
> ```
>
>
> </details>
type X struct {
J *X
K map[string]int
}
### Smaller Encodings with Struct Tag Options
func main() {
raw, _ := hex.DecodeString(data)
decoder := gob.NewDecoder(bytes.NewReader(raw))
Struct tags automatically reduce encoded size of structs and improve speed.
var x X
decoder.Decode(&x) // fatal error: runtime: out of memory
fmt.Println("Decoding finished.")
}
```
<hr/>
</details>
`fxamacker/cbor` is fast at rejecting malformed CBOR data. E.g. attempts to
decode 10 bytes of malicious CBOR data to `[]byte` (with default settings):
| Codec | Speed (ns/op) | Memory | Allocs |
| :---- | ------------: | -----: | -----: |
| fxamacker/cbor 2.5.0 | 44 ± 5% | 32 B/op | 2 allocs/op |
| ugorji/go 1.2.11 | 5353261 ± 4% | 67111321 B/op | 13 allocs/op |
<details><summary>Benchmark details</summary><p/>
Latest comparison used:
- Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
- go1.19.10, linux/amd64, i5-13600K (disabled all e-cores, DDR4 @2933)
- go test -bench=. -benchmem -count=20
#### Prior comparisons
| Codec | Speed (ns/op) | Memory | Allocs |
| :---- | ------------: | -----: | -----: |
| fxamacker/cbor 2.5.0-beta2 | 44.33 ± 2% | 32 B/op | 2 allocs/op |
| fxamacker/cbor 0.1.0 - 2.4.0 | ~44.68 ± 6% | 32 B/op | 2 allocs/op |
| ugorji/go 1.2.10 | 5524792.50 ± 3% | 67110491 B/op | 12 allocs/op |
| ugorji/go 1.1.0 - 1.2.6 | 💥 runtime: | out of memory: | cannot allocate |
- Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
- go1.19.6, linux/amd64, i5-13600K (DDR4)
- go test -bench=. -benchmem -count=20
<hr/>
</details>
### Smaller Encodings with Struct Tags
Struct tags (`toarray`, `keyasint`, `omitempty`) reduce encoded size of structs.
<details><summary>Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
https://go.dev/play/p/YxwvfPdFQG2
```Go
// Example encoding nested struct (with omitempty tag)
// - encoding/json: 18 byte JSON
// - fxamacker/cbor: 1 byte CBOR
package main
import (
"encoding/hex"
"encoding/json"
"fmt"
"github.com/fxamacker/cbor/v2"
)
type GrandChild struct {
Quux int `json:",omitempty"`
}
type Child struct {
Baz int `json:",omitempty"`
Qux GrandChild `json:",omitempty"`
}
type Parent struct {
Foo Child `json:",omitempty"`
Bar int `json:",omitempty"`
}
func cb() {
results, _ := cbor.Marshal(Parent{})
fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
text, _ := cbor.Diagnose(results) // Diagnostic Notation
fmt.Println("DN: " + text)
}
func js() {
results, _ := json.Marshal(Parent{})
fmt.Println("hex(JSON): " + hex.EncodeToString(results))
text := string(results) // JSON
fmt.Println("JSON: " + text)
}
func main() {
cb()
fmt.Println("-------------")
js()
}
```
Output (DN is Diagnostic Notation):
```
hex(CBOR): a0
DN: {}
-------------
hex(JSON): 7b22466f6f223a7b22517578223a7b7d7d7d
JSON: {"Foo":{"Qux":{}}}
```
<hr/>
</details>
Example using different struct tags together:
We can write less code by using struct tag options:
- `toarray`: encode without field names (decode back to original struct)
- `keyasint`: encode field names as integers (decode back to original struct)
- `omitempty`: omit empty fields when encoding
- `omitzero`: omit zero-value fields when encoding
![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags")
API is mostly same as `encoding/json`, plus interfaces that simplify concurrency for CBOR options.
> [!NOTE]
> `fxamacker/cbor` can encode a 3-level nested Go struct to 1 byte!
> - `encoding/json`: 18 bytes of JSON
> - `fxamacker/cbor`: 1 byte of CBOR
>
> <details><summary> 🔎&nbsp; Encoding 3-level nested Go struct with omitempty</summary><p/>
>
> https://go.dev/play/p/YxwvfPdFQG2
>
> ```Go
> // Example encoding nested struct (with omitempty tag)
> // - encoding/json: 18 byte JSON
> // - fxamacker/cbor: 1 byte CBOR
>
> package main
>
> import (
> "encoding/hex"
> "encoding/json"
> "fmt"
>
> "github.com/fxamacker/cbor/v2"
> )
>
> type GrandChild struct {
> Quux int `json:",omitempty"`
> }
>
> type Child struct {
> Baz int `json:",omitempty"`
> Qux GrandChild `json:",omitempty"`
> }
>
> type Parent struct {
> Foo Child `json:",omitempty"`
> Bar int `json:",omitempty"`
> }
>
> func cb() {
> results, _ := cbor.Marshal(Parent{})
> fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
>
> text, _ := cbor.Diagnose(results) // Diagnostic Notation
> fmt.Println("DN: " + text)
> }
>
> func js() {
> results, _ := json.Marshal(Parent{})
> fmt.Println("hex(JSON): " + hex.EncodeToString(results))
>
> text := string(results) // JSON
> fmt.Println("JSON: " + text)
> }
>
> func main() {
> cb()
> fmt.Println("-------------")
> js()
> }
> ```
>
> Output (DN is Diagnostic Notation):
> ```
> hex(CBOR): a0
> DN: {}
> -------------
> hex(JSON): 7b22466f6f223a7b22517578223a7b7d7d7d
> JSON: {"Foo":{"Qux":{}}}
> ```
>
> </details>
## Quick Start
__Install__: `go get github.com/fxamacker/cbor/v2` and `import "github.com/fxamacker/cbor/v2"`.
> [!TIP]
>
> Tinygo users can try beta/experimental branch [feature/cbor-tinygo-beta](https://github.com/fxamacker/cbor/tree/feature/cbor-tinygo-beta).
>
> <details><summary> 🔎&nbsp; More about tinygo feature branch</summary>
>
> ### Tinygo
>
> Branch [feature/cbor-tinygo-beta](https://github.com/fxamacker/cbor/tree/feature/cbor-tinygo-beta) is based on fxamacker/cbor v2.7.0 and it can be compiled using tinygo v0.33 (also compiles with golang/go).
>
> It passes unit tests (with both go1.22 and tinygo v0.33) and is considered beta/experimental for tinygo.
>
> :warning: The `feature/cbor-tinygo-beta` branch does not get fuzz tested yet.
>
> Changes in this feature branch only affect tinygo compiled software. Summary of changes:
> - default `DecOptions.MaxNestedLevels` is reduced to 16 (was 32). User can specify higher limit but 24+ crashes tests when compiled with tinygo v0.33.
> - disabled decoding CBOR tag data to Go interface because tinygo v0.33 is missing needed feature.
> - encoding error message can be different when encoding function type.
>
> Related tinygo issues:
> - https://github.com/tinygo-org/tinygo/issues/4277
> - https://github.com/tinygo-org/tinygo/issues/4458
>
> </details>
### Key Points
This library can encode and decode CBOR (RFC 8949) and CBOR Sequences (RFC 8742).
@ -252,16 +290,17 @@ rest, err = cbor.UnmarshalFirst(b, &v) // decode []byte b to v
// DiagnoseFirst translates first CBOR data item to text and returns remaining bytes.
text, rest, err = cbor.DiagnoseFirst(b) // decode []byte b to Diagnostic Notation text
// NOTE: Unmarshal returns ExtraneousDataError if there are remaining bytes,
// but new funcs UnmarshalFirst and DiagnoseFirst do not.
// NOTE: Unmarshal() returns ExtraneousDataError if there are remaining bytes, but
// UnmarshalFirst() and DiagnoseFirst() allow trailing bytes.
```
__IMPORTANT__: 👉 CBOR settings allow trade-offs between speed, security, encoding size, etc.
- Different CBOR libraries may use different default settings.
- CBOR-based formats or protocols usually require specific settings.
For example, WebAuthn uses "CTAP2 Canonical CBOR" which is available as a preset.
> [!IMPORTANT]
> CBOR settings allow trade-offs between speed, security, encoding size, etc.
>
> - Different CBOR libraries may use different default settings.
> - CBOR-based formats or protocols usually require specific settings.
>
> For example, WebAuthn uses "CTAP2 Canonical CBOR" which is available as a preset.
### Presets
@ -312,9 +351,9 @@ err = em.MarshalToBuffer(v, &buf) // encode v to provided buf
### Struct Tags
Struct tags (`toarray`, `keyasint`, `omitempty`) reduce encoded size of structs.
Struct tag options (`toarray`, `keyasint`, `omitempty`, `omitzero`) reduce encoded size of structs.
<details><summary>Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
<details><summary> 🔎&nbsp; Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
https://go.dev/play/p/YxwvfPdFQG2
@ -382,13 +421,13 @@ JSON: {"Foo":{"Qux":{}}}
</details>
<details><summary>Example using several struct tags</summary><p/>
<details><summary> 🔎&nbsp; Example using struct tag options</summary><p/>
![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags")
</details>
Struct tags simplify use of CBOR-based protocols that require CBOR arrays or maps with integer keys.
Struct tag options simplify use of CBOR-based protocols that require CBOR arrays or maps with integer keys.
### CBOR Tags
@ -404,7 +443,7 @@ em, err := opts.EncModeWithSharedTags(ts) // mutable shared CBOR tags
`TagSet` and modes using it are safe for concurrent use. Equivalent API is available for `DecMode`.
<details><summary>Example using TagSet and TagOptions</summary><p/>
<details><summary> 🔎&nbsp; Example using TagSet and TagOptions</summary><p/>
```go
// Use signedCWT struct defined in "Decoding CWT" example.
@ -430,7 +469,7 @@ if err := dm.Unmarshal(data, &v); err != nil {
em, _ := cbor.EncOptions{}.EncModeWithTags(tags)
// Marshal signedCWT with tag number.
if data, err := cbor.Marshal(v); err != nil {
if data, err := em.Marshal(v); err != nil {
return err
}
```
@ -439,7 +478,7 @@ if data, err := cbor.Marshal(v); err != nil {
### Functions and Interfaces
<details><summary>Functions and interfaces at a glance</summary><p/>
<details><summary> 🔎&nbsp; Functions and interfaces at a glance</summary><p/>
Common functions with same API as `encoding/json`:
- `Marshal`, `Unmarshal`
@ -472,11 +511,24 @@ Default limits may need to be increased for systems handling very large data (e.
## Status
v2.7.0 (June 23, 2024) adds features and improvements that help large projects (e.g. Kubernetes) use CBOR as an alternative to JSON and Protocol Buffers. Other improvements include speedups, improved memory use, bug fixes, new serialization options, etc. It passed fuzz tests (5+ billion executions) and is production quality.
v2.8.0 (March 30, 2025) is a small release primarily to add `omitzero` option to struct field tags and fix bugs. It passed fuzz tests (billions of executions) and is production quality.
v2.8.0 and v2.7.1 fixes these 3 functions (when called directly by user apps) to use same error handling on bad inputs as `cbor.Unmarshal()`:
- `ByteString.UnmarshalCBOR()`
- `RawTag.UnmarshalCBOR()`
- `SimpleValue.UnmarshalCBOR()`
The above 3 `UnmarshalCBOR()` functions were initially created for internal use and are deprecated now, so please use `Unmarshal()` or `UnmarshalFirst()` instead. To preserve backward compatibility, these deprecated functions were added to fuzz tests and will not be removed in v2.
The minimum version of Go required to build:
- v2.8.0 requires go 1.20.
- v2.7.1 and older releases require go 1.17.
For more details, see [release notes](https://github.com/fxamacker/cbor/releases).
### Prior Release
### Prior Releases
v2.7.0 (June 23, 2024) adds features and improvements that help large projects (e.g. Kubernetes) use CBOR as an alternative to JSON and Protocol Buffers. Other improvements include speedups, improved memory use, bug fixes, new serialization options, etc. It passed fuzz tests (5+ billion executions) and is production quality.
[v2.6.0](https://github.com/fxamacker/cbor/releases/tag/v2.6.0) (February 2024) adds important new features, optimizations, and bug fixes. It is especially useful to systems that need to convert data between CBOR and JSON. New options and optimizations improve handling of bignum, integers, maps, and strings.
@ -489,7 +541,7 @@ See [v2.5.0 release notes](https://github.com/fxamacker/cbor/releases/tag/v2.5.0
See ["Version and API Changes"](https://github.com/fxamacker/cbor#versions-and-api-changes) section for more info about version numbering, etc.
<!--
<details><summary>👉 Benchmark Comparison: v2.4.0 vs v2.5.0</summary><p/>
<details><summary> 🔎&nbsp; Benchmark Comparison: v2.4.0 vs v2.5.0</summary><p/>
TODO: Update to v2.4.0 vs 2.5.0 (not beta2).
@ -549,7 +601,7 @@ geomean 2.782
## Who uses fxamacker/cbor
`fxamacker/cbor` is used in projects by Arm Ltd., Berlin Institute of Health at Charité, Chainlink, Cisco, Confidential Computing Consortium, ConsenSys, Dapper&nbsp;Labs, EdgeX&nbsp;Foundry, F5, FIDO Alliance, Fraunhofer&#8209;AISEC, Kubernetes, Let's Encrypt (ISRG), Linux&nbsp;Foundation, Matrix.org, Microsoft, Mozilla, National&nbsp;Cybersecurity&nbsp;Agency&nbsp;of&nbsp;France (govt), Netherlands (govt), Oasis Protocol, Smallstep, Tailscale, Taurus SA, Teleport, TIBCO, and others.
`fxamacker/cbor` is used in projects by Arm Ltd., Berlin Institute of Health at Charité, Chainlink, Cisco, Confidential&nbsp;Computing&nbsp;Consortium, ConsenSys, EdgeX&nbsp;Foundry, F5, Flow&nbsp;Foundation, Fraunhofer&#8209;AISEC, IBM, Kubernetes, Let's&nbsp;Encrypt&nbsp;(ISRG), Linux&nbsp;Foundation, Matrix.org, Microsoft, Mozilla, National&nbsp;Cybersecurity&nbsp;Agency&nbsp;of&nbsp;France&nbsp;(govt), Netherlands&nbsp;(govt), Oasis&nbsp;Protocol, Smallstep, Tailscale, Taurus SA, Teleport, TIBCO, and others.
`fxamacker/cbor` passed multiple confidential security assessments. A [nonconfidential security assessment](https://github.com/veraison/go-cose/blob/v1.0.0-rc.1/reports/NCC_Microsoft-go-cose-Report_2022-05-26_v1.0.pdf) (prepared by NCC Group for Microsoft Corporation) includes a subset of fxamacker/cbor v2.4.0 in its scope.
@ -588,7 +640,7 @@ By default, decoder treats time values of floating-point NaN and Infinity as if
__Click to expand topic:__
<details>
<summary>Duplicate Map Keys</summary><p>
<summary> 🔎&nbsp; Duplicate Map Keys</summary><p>
This library provides options for fast detection and rejection of duplicate map keys based on applying a Go-specific data model to CBOR's extended generic data model in order to determine duplicate vs distinct map keys. Detection relies on whether the CBOR map key would be a duplicate "key" when decoded and applied to the user-provided Go map or struct.
@ -601,7 +653,7 @@ APF suffix means "Allow Partial Fill" so the destination map or struct can conta
</details>
<details>
<summary>Tag Validity</summary><p>
<summary> 🔎&nbsp; Tag Validity</summary><p>
This library checks tag validity for built-in tags (currently tag numbers 0, 1, 2, 3, and 55799):

View File

@ -188,16 +188,16 @@ var decodeBenchmarks = []struct {
var encodeBenchmarks = []struct {
name string
data []byte
values []interface{}
values []any
}{
{"bool", hexDecode("f5"), []interface{}{true}},
{"positive int", hexDecode("1bffffffffffffffff"), []interface{}{uint64(18446744073709551615)}},
{"negative int", hexDecode("3903e7"), []interface{}{int64(-1000)}},
{"float", hexDecode("fbc010666666666666"), []interface{}{float64(-4.1)}},
{"bytes", hexDecode("581a0102030405060708090a0b0c0d0e0f101112131415161718191a"), []interface{}{[]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}}},
{"text", hexDecode("782b54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67"), []interface{}{"The quick brown fox jumps over the lazy dog"}},
{"array", hexDecode("981a0102030405060708090a0b0c0d0e0f101112131415161718181819181a"), []interface{}{[]int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}}},
{"map", hexDecode("ad616161416162614261636143616461446165614561666146616761476168614861696149616a614a616c614c616d614d616e614e"), []interface{}{map[string]string{"a": "A", "b": "B", "c": "C", "d": "D", "e": "E", "f": "F", "g": "G", "h": "H", "i": "I", "j": "J", "l": "L", "m": "M", "n": "N"}}},
{"bool", hexDecode("f5"), []any{true}},
{"positive int", hexDecode("1bffffffffffffffff"), []any{uint64(18446744073709551615)}},
{"negative int", hexDecode("3903e7"), []any{int64(-1000)}},
{"float", hexDecode("fbc010666666666666"), []any{float64(-4.1)}},
{"bytes", hexDecode("581a0102030405060708090a0b0c0d0e0f101112131415161718191a"), []any{[]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}}},
{"text", hexDecode("782b54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67"), []any{"The quick brown fox jumps over the lazy dog"}},
{"array", hexDecode("981a0102030405060708090a0b0c0d0e0f101112131415161718181819181a"), []any{[]int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}}},
{"map", hexDecode("ad616161416162614261636143616461446165614561666146616761476168614861696149616a614a616c614c616d614d616e614e"), []any{map[string]string{"a": "A", "b": "B", "c": "C", "d": "D", "e": "E", "f": "F", "g": "G", "h": "H", "i": "I", "j": "J", "l": "L", "m": "M", "n": "N"}}},
}
func BenchmarkUnmarshal(b *testing.B) {
@ -226,7 +226,7 @@ func BenchmarkUnmarshal(b *testing.B) {
{
"CBOR map to Go map[string]interface{}",
hexDecode("a86154f56255691bffffffffffffffff61493903e76146fbc0106666666666666142581a0102030405060708090a0b0c0d0e0f101112131415161718191a6153782b54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f6764536c6369981a0102030405060708090a0b0c0d0e0f101112131415161718181819181a634d7373ad6163614361656145616661466167614761686148616e614e616d614d61616141616261426164614461696149616a614a616c614c"),
reflect.TypeOf(map[string]interface{}{}),
reflect.TypeOf(map[string]any{}),
},
// Unmarshal CBOR map with string key to struct.
{
@ -238,7 +238,7 @@ func BenchmarkUnmarshal(b *testing.B) {
{
"CBOR map to Go map[int]interface{}",
hexDecode("a801f5021bffffffffffffffff033903e704fbc01066666666666605581a0102030405060708090a0b0c0d0e0f101112131415161718191a06782b54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f6707981a0102030405060708090a0b0c0d0e0f101112131415161718181819181a08ad61646144616661466167614761686148616d614d616e614e6161614161626142616361436165614561696149616a614a616c614c"),
reflect.TypeOf(map[int]interface{}{}),
reflect.TypeOf(map[int]any{}),
},
// Unmarshal CBOR map with integer key, such as COSE Key and SenML, to struct.
{
@ -250,7 +250,7 @@ func BenchmarkUnmarshal(b *testing.B) {
{
"CBOR array to Go []interface{}",
hexDecode("88f51bffffffffffffffff3903e7fbc010666666666666581a0102030405060708090a0b0c0d0e0f101112131415161718191a782b54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67981a0102030405060708090a0b0c0d0e0f101112131415161718181819181aad616261426163614361646144616561456166614661696149616e614e616161416167614761686148616a614a616c614c616d614d"),
reflect.TypeOf([]interface{}{}),
reflect.TypeOf([]any{}),
},
// Unmarshal CBOR array of known sequence of data types, such as signed/maced/encrypted CWT, to struct.
{
@ -384,7 +384,7 @@ func BenchmarkMarshal(b *testing.B) {
}
}
// Marshal map[string]interface{} to CBOR map
m1 := map[string]interface{}{ //nolint:dupl
m1 := map[string]any{
"T": true,
"UI": uint(18446744073709551615),
"I": -1000,
@ -406,7 +406,7 @@ func BenchmarkMarshal(b *testing.B) {
Mss: map[string]string{"a": "A", "b": "B", "c": "C", "d": "D", "e": "E", "f": "F", "g": "G", "h": "H", "i": "I", "j": "J", "l": "L", "m": "M", "n": "N"},
}
// Marshal map[int]interface{} to CBOR map
m2 := map[int]interface{}{ //nolint:dupl
m2 := map[int]any{
1: true,
2: uint(18446744073709551615),
3: -1000,
@ -428,7 +428,7 @@ func BenchmarkMarshal(b *testing.B) {
Mss: map[string]string{"a": "A", "b": "B", "c": "C", "d": "D", "e": "E", "f": "F", "g": "G", "h": "H", "i": "I", "j": "J", "l": "L", "m": "M", "n": "N"},
}
// Marshal []interface to CBOR array.
slc := []interface{}{
slc := []any{
true,
uint(18446744073709551615),
-1000,
@ -451,7 +451,7 @@ func BenchmarkMarshal(b *testing.B) {
}
var moreBenchmarks = []struct {
name string
value interface{}
value any
}{
{"Go map[string]interface{} to CBOR map", m1},
{"Go struct to CBOR map", v1},
@ -501,9 +501,9 @@ func BenchmarkMarshalCanonical(b *testing.B) {
for _, bm := range []struct {
name string
data []byte
values []interface{}
values []any
}{
{"map", hexDecode("ad616161416162614261636143616461446165614561666146616761476168614861696149616a614a616c614c616d614d616e614e"), []interface{}{map[string]string{"a": "A", "b": "B", "c": "C", "d": "D", "e": "E", "f": "F", "g": "G", "h": "H", "i": "I", "j": "J", "l": "L", "m": "M", "n": "N"}, strc{A: "A", B: "B", C: "C", D: "D", E: "E", F: "F", G: "G", H: "H", I: "I", J: "J", L: "L", M: "M", N: "N"}, map[int]int{0: 0} /* single-entry map */}},
{"map", hexDecode("ad616161416162614261636143616461446165614561666146616761476168614861696149616a614a616c614c616d614d616e614e"), []any{map[string]string{"a": "A", "b": "B", "c": "C", "d": "D", "e": "E", "f": "F", "g": "G", "h": "H", "i": "I", "j": "J", "l": "L", "m": "M", "n": "N"}, strc{A: "A", B: "B", C: "C", D: "D", E: "E", F: "F", G: "G", H: "H", I: "I", J: "J", L: "L", M: "M", N: "N"}, map[int]int{0: 0} /* single-entry map */}},
} {
for _, v := range bm.values {
name := "Go " + reflect.TypeOf(v).String() + " to CBOR " + bm.name
@ -860,7 +860,7 @@ func BenchmarkUnmarshalMapToStruct(b *testing.B) {
type input struct {
name string
data []byte
into interface{}
into any
reject bool
}

View File

@ -38,11 +38,38 @@ func (bs ByteString) MarshalCBOR() ([]byte, error) {
// UnmarshalCBOR decodes CBOR byte string (major type 2) to ByteString.
// Decoding CBOR null and CBOR undefined sets ByteString to be empty.
//
// Deprecated: No longer used by this codec; kept for compatibility
// with user apps that directly call this function.
func (bs *ByteString) UnmarshalCBOR(data []byte) error {
if bs == nil {
return errors.New("cbor.ByteString: UnmarshalCBOR on nil pointer")
}
d := decoder{data: data, dm: defaultDecMode}
// Check well-formedness of CBOR data item.
// ByteString.UnmarshalCBOR() is exported, so
// the codec needs to support same behavior for:
// - Unmarshal(data, *ByteString)
// - ByteString.UnmarshalCBOR(data)
err := d.wellformed(false, false)
if err != nil {
return err
}
return bs.unmarshalCBOR(data)
}
// unmarshalCBOR decodes CBOR byte string (major type 2) to ByteString.
// Decoding CBOR null and CBOR undefined sets ByteString to be empty.
// This function assumes data is well-formed, and does not perform bounds checking.
// This function is called by Unmarshal().
func (bs *ByteString) unmarshalCBOR(data []byte) error {
if bs == nil {
return errors.New("cbor.ByteString: UnmarshalCBOR on nil pointer")
}
// Decoding CBOR null and CBOR undefined to ByteString resets data.
// This behavior is similar to decoding CBOR null and CBOR undefined to []byte.
if len(data) == 1 && (data[0] == 0xf6 || data[0] == 0xf7) {

View File

@ -3,7 +3,11 @@
package cbor
import "testing"
import (
"io"
"strings"
"testing"
)
func TestByteString(t *testing.T) {
type s1 struct {
@ -99,3 +103,110 @@ func TestByteString(t *testing.T) {
dm, _ := DecOptions{}.DecMode()
testRoundTrip(t, testCases, em, dm)
}
func TestUnmarshalByteStringOnBadData(t *testing.T) {
testCases := []struct {
name string
data []byte
errMsg string
}{
// Empty data
{
name: "nil data",
data: nil,
errMsg: io.EOF.Error(),
},
{
name: "empty data",
data: []byte{},
errMsg: io.EOF.Error(),
},
// Wrong CBOR types
{
name: "uint type",
data: hexDecode("01"),
errMsg: "cbor: cannot unmarshal positive integer into Go value of type cbor.ByteString",
},
{
name: "int type",
data: hexDecode("20"),
errMsg: "cbor: cannot unmarshal negative integer into Go value of type cbor.ByteString",
},
{
name: "string type",
data: hexDecode("60"),
errMsg: "cbor: cannot unmarshal UTF-8 text string into Go value of type cbor.ByteString",
},
{
name: "array type",
data: hexDecode("80"),
errMsg: "cbor: cannot unmarshal array into Go value of type cbor.ByteString",
},
{
name: "map type",
data: hexDecode("a0"),
errMsg: "cbor: cannot unmarshal map into Go value of type cbor.ByteString",
},
{
name: "tag type",
data: hexDecode("c074323031332d30332d32315432303a30343a30305a"),
errMsg: "cbor: cannot unmarshal tag into Go value of type cbor.ByteString",
},
{
name: "float type",
data: hexDecode("f90000"),
errMsg: "cbor: cannot unmarshal primitives into Go value of type cbor.ByteString",
},
// Truncated CBOR data
{
name: "truncated head",
data: hexDecode("18"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
// Truncated CBOR byte string
{
name: "truncated byte string",
data: hexDecode("44010203"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
// Extraneous CBOR data
{
name: "extraneous data",
data: hexDecode("c074323031332d30332d32315432303a30343a30305a00"),
errMsg: "cbor: 1 bytes of extraneous data starting at index 22",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Test ByteString.UnmarshalCBOR(data)
{
var v ByteString
err := v.UnmarshalCBOR(tc.data)
if err == nil {
t.Errorf("UnmarshalCBOR(%x) didn't return error", tc.data)
}
if !strings.HasPrefix(err.Error(), tc.errMsg) {
t.Errorf("UnmarshalCBOR(%x) returned error %q, want %q", tc.data, err.Error(), tc.errMsg)
}
}
// Test Unmarshal(data, *ByteString), which calls ByteString.unmarshalCBOR() under the hood
{
var v ByteString
err := Unmarshal(tc.data, &v)
if err == nil {
t.Errorf("Unmarshal(%x) didn't return error", tc.data)
}
if !strings.HasPrefix(err.Error(), tc.errMsg) {
t.Errorf("Unmarshal(%x) returned error %q, want %q", tc.data, err.Error(), tc.errMsg)
}
}
})
}
}

View File

@ -17,6 +17,7 @@ import (
type encodeFuncs struct {
ef encodeFunc
ief isEmptyFunc
izf isZeroFunc
}
var (
@ -31,6 +32,7 @@ type specialType int
const (
specialTypeNone specialType = iota
specialTypeUnmarshalerIface
specialTypeUnexportedUnmarshalerIface
specialTypeEmptyIface
specialTypeIface
specialTypeTag
@ -50,7 +52,7 @@ type typeInfo struct {
func newTypeInfo(t reflect.Type) *typeInfo {
tInfo := typeInfo{typ: t, kind: t.Kind()}
for t.Kind() == reflect.Ptr {
for t.Kind() == reflect.Pointer {
t = t.Elem()
}
@ -69,7 +71,9 @@ func newTypeInfo(t reflect.Type) *typeInfo {
tInfo.spclType = specialTypeTag
} else if t == typeTime {
tInfo.spclType = specialTypeTime
} else if reflect.PtrTo(t).Implements(typeUnmarshaler) {
} else if reflect.PointerTo(t).Implements(typeUnexportedUnmarshaler) {
tInfo.spclType = specialTypeUnexportedUnmarshalerIface
} else if reflect.PointerTo(t).Implements(typeUnmarshaler) {
tInfo.spclType = specialTypeUnmarshalerIface
}
@ -237,7 +241,7 @@ func getEncodingStructType(t reflect.Type) (*encodingStructType, error) {
e := getEncodeBuffer()
for i := 0; i < len(flds); i++ {
// Get field's encodeFunc
flds[i].ef, flds[i].ief = getEncodeFunc(flds[i].typ)
flds[i].ef, flds[i].ief, flds[i].izf = getEncodeFunc(flds[i].typ)
if flds[i].ef == nil {
err = &UnsupportedTypeError{t}
break
@ -321,7 +325,7 @@ func getEncodingStructType(t reflect.Type) (*encodingStructType, error) {
func getEncodingStructToArrayType(t reflect.Type, flds fields) (*encodingStructType, error) {
for i := 0; i < len(flds); i++ {
// Get field's encodeFunc
flds[i].ef, flds[i].ief = getEncodeFunc(flds[i].typ)
flds[i].ef, flds[i].ief, flds[i].izf = getEncodeFunc(flds[i].typ)
if flds[i].ef == nil {
structType := &encodingStructType{err: &UnsupportedTypeError{t}}
encodingStructTypeCache.Store(t, structType)
@ -337,14 +341,14 @@ func getEncodingStructToArrayType(t reflect.Type, flds fields) (*encodingStructT
return structType, structType.err
}
func getEncodeFunc(t reflect.Type) (encodeFunc, isEmptyFunc) {
func getEncodeFunc(t reflect.Type) (encodeFunc, isEmptyFunc, isZeroFunc) {
if v, _ := encodeFuncCache.Load(t); v != nil {
fs := v.(encodeFuncs)
return fs.ef, fs.ief
return fs.ef, fs.ief, fs.izf
}
ef, ief := getEncodeFuncInternal(t)
encodeFuncCache.Store(t, encodeFuncs{ef, ief})
return ef, ief
ef, ief, izf := getEncodeFuncInternal(t)
encodeFuncCache.Store(t, encodeFuncs{ef, ief, izf})
return ef, ief, izf
}
func getTypeInfo(t reflect.Type) *typeInfo {

147
decode.go
View File

@ -13,6 +13,7 @@ import (
"io"
"math"
"math/big"
"math/bits"
"reflect"
"strconv"
"strings"
@ -104,7 +105,7 @@ import (
// if there are any remaining bytes following the first valid CBOR data item.
// See UnmarshalFirst, if you want to unmarshal only the first
// CBOR data item without ExtraneousDataError caused by remaining bytes.
func Unmarshal(data []byte, v interface{}) error {
func Unmarshal(data []byte, v any) error {
return defaultDecMode.Unmarshal(data, v)
}
@ -114,7 +115,7 @@ func Unmarshal(data []byte, v interface{}) error {
// If v is nil, not a pointer, or a nil pointer, UnmarshalFirst returns an error.
//
// See the documentation for Unmarshal for details.
func UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error) {
func UnmarshalFirst(data []byte, v any) (rest []byte, err error) {
return defaultDecMode.UnmarshalFirst(data, v)
}
@ -151,6 +152,10 @@ type Unmarshaler interface {
UnmarshalCBOR([]byte) error
}
type unmarshaler interface {
unmarshalCBOR([]byte) error
}
// InvalidUnmarshalError describes an invalid argument passed to Unmarshal.
type InvalidUnmarshalError struct {
s string
@ -193,7 +198,7 @@ func (e *InvalidMapKeyTypeError) Error() string {
// DupMapKeyError describes detected duplicate map key in CBOR map.
type DupMapKeyError struct {
Key interface{}
Key any
Index int
}
@ -780,11 +785,13 @@ type DecOptions struct {
MaxNestedLevels int
// MaxArrayElements specifies the max number of elements for CBOR arrays.
// Default is 128*1024=131072 and it can be set to [16, 2147483647]
// Default is 128*1024=131072 and it can be set to [16, 2147483647] on 32-bit arch
// or [16, 9223372036854775807] on 64-bit arch.
MaxArrayElements int
// MaxMapPairs specifies the max number of key-value pairs for CBOR maps.
// Default is 128*1024=131072 and it can be set to [16, 2147483647]
// Default is 128*1024=131072 and it can be set to [16, 2147483647] on 32-bit arch
// or [16, 9223372036854775807] on 64-bit arch.
MaxMapPairs int
// IndefLength specifies whether to allow indefinite length CBOR items.
@ -953,13 +960,15 @@ func (opts DecOptions) DecModeWithSharedTags(tags TagSet) (DecMode, error) { //n
}
const (
maxInt = 1<<(bits.UintSize-1) - 1
defaultMaxArrayElements = 131072
minMaxArrayElements = 16
maxMaxArrayElements = 2147483647
maxMaxArrayElements = maxInt
defaultMaxMapPairs = 131072
minMaxMapPairs = 16
maxMaxMapPairs = 2147483647
maxMaxMapPairs = maxInt
defaultMaxNestedLevels = 32
minMaxNestedLevels = 4
@ -1130,7 +1139,7 @@ type DecMode interface {
// Unmarshal returns an error.
//
// See the documentation for Unmarshal for details.
Unmarshal(data []byte, v interface{}) error
Unmarshal(data []byte, v any) error
// UnmarshalFirst parses the first CBOR data item into the value pointed to by v
// using the decoding mode. Any remaining bytes are returned in rest.
@ -1138,7 +1147,7 @@ type DecMode interface {
// If v is nil, not a pointer, or a nil pointer, UnmarshalFirst returns an error.
//
// See the documentation for Unmarshal for details.
UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error)
UnmarshalFirst(data []byte, v any) (rest []byte, err error)
// Valid checks whether data is a well-formed encoded CBOR data item and
// that it complies with configurable restrictions such as MaxNestedLevels,
@ -1165,6 +1174,12 @@ type DecMode interface {
// NewDecoder returns a new decoder that reads from r using dm DecMode.
NewDecoder(r io.Reader) *Decoder
// NewStreamDecoder returns a new StreamDecoder that reads from r using dm DecMode.
NewStreamDecoder(r io.Reader) *StreamDecoder
// NewByteStreamDecoder returns a new StreamDecoder that reads from data using dm DecMode.
NewByteStreamDecoder(data []byte) *StreamDecoder
// DecOptions returns user specified options used to create this DecMode.
DecOptions() DecOptions
}
@ -1245,7 +1260,7 @@ func (dm *decMode) DecOptions() DecOptions {
// Unmarshal returns an error.
//
// See the documentation for Unmarshal for details.
func (dm *decMode) Unmarshal(data []byte, v interface{}) error {
func (dm *decMode) Unmarshal(data []byte, v any) error {
d := decoder{data: data, dm: dm}
// Check well-formedness.
@ -1265,7 +1280,7 @@ func (dm *decMode) Unmarshal(data []byte, v interface{}) error {
// If v is nil, not a pointer, or a nil pointer, UnmarshalFirst returns an error.
//
// See the documentation for Unmarshal for details.
func (dm *decMode) UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error) {
func (dm *decMode) UnmarshalFirst(data []byte, v any) (rest []byte, err error) {
d := decoder{data: data, dm: dm}
// check well-formedness.
@ -1317,7 +1332,25 @@ func (dm *decMode) Wellformed(data []byte) error {
// NewDecoder returns a new decoder that reads from r using dm DecMode.
func (dm *decMode) NewDecoder(r io.Reader) *Decoder {
return &Decoder{r: r, d: decoder{dm: dm}}
return &Decoder{r: r, d: &decoder{dm: dm}}
}
// NewStreamDecoder returns a new StreamDecoder that reads from r using dm DecMode.
func (dm *decMode) NewStreamDecoder(r io.Reader) *StreamDecoder {
return &StreamDecoder{
dec: &Decoder{r: r, d: &decoder{dm: dm}},
}
}
// NewByteStreamDecoder returns a new StreamDecoder that reads from data using dm DecMode.
func (dm *decMode) NewByteStreamDecoder(data []byte) *StreamDecoder {
return &StreamDecoder{
dec: &Decoder{
r: nil,
d: &decoder{dm: dm},
buf: data,
},
}
}
type decoder struct {
@ -1341,13 +1374,13 @@ type decoder struct {
// If CBOR data item fails to be decoded into v,
// error is returned and offset is moved to the next CBOR data item.
// Precondition: d.data contains at least one well-formed CBOR data item.
func (d *decoder) value(v interface{}) error {
func (d *decoder) value(v any) error {
// v can't be nil, non-pointer, or nil pointer value.
if v == nil {
return &InvalidUnmarshalError{"cbor: Unmarshal(nil)"}
}
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr {
if rv.Kind() != reflect.Pointer {
return &InvalidUnmarshalError{"cbor: Unmarshal(non-pointer " + rv.Type().String() + ")"}
} else if rv.IsNil() {
return &InvalidUnmarshalError{"cbor: Unmarshal(nil " + rv.Type().String() + ")"}
@ -1361,7 +1394,7 @@ func (d *decoder) value(v interface{}) error {
func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolint:gocyclo
// Decode CBOR nil or CBOR undefined to pointer value by setting pointer value to nil.
if d.nextCBORNil() && v.Kind() == reflect.Ptr {
if d.nextCBORNil() && v.Kind() == reflect.Pointer {
d.skip()
v.Set(reflect.Zero(v.Type()))
return nil
@ -1387,7 +1420,7 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
registeredType := d.dm.tags.getTypeFromTagNum(tagNums)
if registeredType != nil {
if registeredType.Implements(tInfo.nonPtrType) ||
reflect.PtrTo(registeredType).Implements(tInfo.nonPtrType) {
reflect.PointerTo(registeredType).Implements(tInfo.nonPtrType) {
v.Set(reflect.New(registeredType))
v = v.Elem()
tInfo = getTypeInfo(registeredType)
@ -1399,7 +1432,7 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
// Create new value for the pointer v to point to.
// At this point, CBOR value is not nil/undefined if v is a pointer.
for v.Kind() == reflect.Ptr {
for v.Kind() == reflect.Pointer {
if v.IsNil() {
if !v.CanSet() {
d.skip()
@ -1460,6 +1493,9 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
case specialTypeUnmarshalerIface:
return d.parseToUnmarshaler(v)
case specialTypeUnexportedUnmarshalerIface:
return d.parseToUnexportedUnmarshaler(v)
}
}
@ -1788,12 +1824,12 @@ func (d *decoder) parseToTime() (time.Time, bool, error) {
// parseToUnmarshaler parses CBOR data to value implementing Unmarshaler interface.
// It assumes data is well-formed, and does not perform bounds checking.
func (d *decoder) parseToUnmarshaler(v reflect.Value) error {
if d.nextCBORNil() && v.Kind() == reflect.Ptr && v.IsNil() {
if d.nextCBORNil() && v.Kind() == reflect.Pointer && v.IsNil() {
d.skip()
return nil
}
if v.Kind() != reflect.Ptr && v.CanAddr() {
if v.Kind() != reflect.Pointer && v.CanAddr() {
v = v.Addr()
}
if u, ok := v.Interface().(Unmarshaler); ok {
@ -1805,9 +1841,29 @@ func (d *decoder) parseToUnmarshaler(v reflect.Value) error {
return errors.New("cbor: failed to assert " + v.Type().String() + " as cbor.Unmarshaler")
}
// parseToUnexportedUnmarshaler parses CBOR data to value implementing unmarshaler interface.
// It assumes data is well-formed, and does not perform bounds checking.
func (d *decoder) parseToUnexportedUnmarshaler(v reflect.Value) error {
if d.nextCBORNil() && v.Kind() == reflect.Pointer && v.IsNil() {
d.skip()
return nil
}
if v.Kind() != reflect.Pointer && v.CanAddr() {
v = v.Addr()
}
if u, ok := v.Interface().(unmarshaler); ok {
start := d.off
d.skip()
return u.unmarshalCBOR(d.data[start:d.off])
}
d.skip()
return errors.New("cbor: failed to assert " + v.Type().String() + " as cbor.unmarshaler")
}
// parse parses CBOR data and returns value in default Go type.
// It assumes data is well-formed, and does not perform bounds checking.
func (d *decoder) parse(skipSelfDescribedTag bool) (interface{}, error) { //nolint:gocyclo
func (d *decoder) parse(skipSelfDescribedTag bool) (any, error) { //nolint:gocyclo
// Strip self-described CBOR tag number.
if skipSelfDescribedTag {
for d.nextCBORType() == cborTypeTag {
@ -2224,15 +2280,15 @@ func (d *decoder) parseTextString() ([]byte, error) {
return b, nil
}
func (d *decoder) parseArray() ([]interface{}, error) {
func (d *decoder) parseArray() ([]any, error) {
_, _, val, indefiniteLength := d.getHeadWithIndefiniteLengthFlag()
hasSize := !indefiniteLength
count := int(val)
if !hasSize {
count = d.numOfItemsUntilBreak() // peek ahead to get array size to preallocate slice for better performance
}
v := make([]interface{}, count)
var e interface{}
v := make([]any, count)
var e any
var err, lastErr error
for i := 0; (hasSize && i < count) || (!hasSize && !d.foundBreak()); i++ {
if e, lastErr = d.parse(true); lastErr != nil {
@ -2298,12 +2354,12 @@ func (d *decoder) parseArrayToArray(v reflect.Value, tInfo *typeInfo) error {
return err
}
func (d *decoder) parseMap() (interface{}, error) {
func (d *decoder) parseMap() (any, error) {
_, _, val, indefiniteLength := d.getHeadWithIndefiniteLengthFlag()
hasSize := !indefiniteLength
count := int(val)
m := make(map[interface{}]interface{})
var k, e interface{}
m := make(map[any]any)
var k, e any
var err, lastErr error
keyCount := 0
for i := 0; (hasSize && i < count) || (!hasSize && !d.foundBreak()); i++ {
@ -2368,11 +2424,7 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
hasSize := !indefiniteLength
count := int(val)
if v.IsNil() {
mapsize := count
if !hasSize {
mapsize = 0
}
v.Set(reflect.MakeMapWithSize(tInfo.nonPtrType, mapsize))
v.Set(reflect.MakeMap(tInfo.nonPtrType))
}
keyType, eleType := tInfo.keyTypeInfo.typ, tInfo.elemTypeInfo.typ
reuseKey, reuseEle := isImmutableKind(tInfo.keyTypeInfo.kind), isImmutableKind(tInfo.elemTypeInfo.kind)
@ -2380,9 +2432,9 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
keyIsInterfaceType := keyType == typeIntf // If key type is interface{}, need to check if key value is hashable.
var err, lastErr error
keyCount := v.Len()
var existingKeys map[interface{}]bool // Store existing map keys, used for detecting duplicate map key.
var existingKeys map[any]bool // Store existing map keys, used for detecting duplicate map key.
if d.dm.dupMapKey == DupMapKeyEnforcedAPF {
existingKeys = make(map[interface{}]bool, keyCount)
existingKeys = make(map[any]bool, keyCount)
if keyCount > 0 {
vKeys := v.MapKeys()
for i := 0; i < len(vKeys); i++ {
@ -2413,7 +2465,7 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
if !isHashableValue(keyValue.Elem()) {
var converted bool
if d.dm.mapKeyByteString == MapKeyByteStringAllowed {
var k interface{}
var k any
k, converted = convertByteSliceToByteString(keyValue.Elem().Interface())
if converted {
keyValue.Set(reflect.ValueOf(k))
@ -2584,7 +2636,7 @@ func (d *decoder) parseMapToStruct(v reflect.Value, tInfo *typeInfo) error { //n
// Keeps track of CBOR map keys to detect duplicate map key
keyCount := 0
var mapKeys map[interface{}]struct{}
var mapKeys map[any]struct{}
errOnUnknownField := (d.dm.extraReturnErrors & ExtraDecErrorUnknownField) > 0
@ -2594,7 +2646,7 @@ MapEntryLoop:
// If duplicate field detection is enabled and the key at index j did not match any
// field, k will hold the map key.
var k interface{}
var k any
t := d.nextCBORType()
if t == cborTypeTextString || (t == cborTypeByteString && d.dm.fieldNameByteString == FieldNameByteStringAllowed) {
@ -2764,7 +2816,7 @@ MapEntryLoop:
// check is never reached.
if d.dm.dupMapKey == DupMapKeyEnforcedAPF {
if mapKeys == nil {
mapKeys = make(map[interface{}]struct{}, 1)
mapKeys = make(map[any]struct{}, 1)
}
mapKeys[k] = struct{}{}
newKeyCount := len(mapKeys)
@ -2969,18 +3021,19 @@ func (d *decoder) nextCBORNil() bool {
}
var (
typeIntf = reflect.TypeOf([]interface{}(nil)).Elem()
typeTime = reflect.TypeOf(time.Time{})
typeBigInt = reflect.TypeOf(big.Int{})
typeUnmarshaler = reflect.TypeOf((*Unmarshaler)(nil)).Elem()
typeBinaryUnmarshaler = reflect.TypeOf((*encoding.BinaryUnmarshaler)(nil)).Elem()
typeString = reflect.TypeOf("")
typeByteSlice = reflect.TypeOf([]byte(nil))
typeIntf = reflect.TypeOf([]any(nil)).Elem()
typeTime = reflect.TypeOf(time.Time{})
typeBigInt = reflect.TypeOf(big.Int{})
typeUnmarshaler = reflect.TypeOf((*Unmarshaler)(nil)).Elem()
typeUnexportedUnmarshaler = reflect.TypeOf((*unmarshaler)(nil)).Elem()
typeBinaryUnmarshaler = reflect.TypeOf((*encoding.BinaryUnmarshaler)(nil)).Elem()
typeString = reflect.TypeOf("")
typeByteSlice = reflect.TypeOf([]byte(nil))
)
func fillNil(_ cborType, v reflect.Value) error {
switch v.Kind() {
case reflect.Slice, reflect.Map, reflect.Interface, reflect.Ptr:
case reflect.Slice, reflect.Map, reflect.Interface, reflect.Pointer:
v.Set(reflect.Zero(v.Type()))
return nil
}
@ -3083,7 +3136,7 @@ func fillFloat(t cborType, val float64, v reflect.Value) error {
}
func fillByteString(t cborType, val []byte, shared bool, v reflect.Value, bsts ByteStringToStringMode, bum BinaryUnmarshalerMode) error {
if bum == BinaryUnmarshalerByteString && reflect.PtrTo(v.Type()).Implements(typeBinaryUnmarshaler) {
if bum == BinaryUnmarshalerByteString && reflect.PointerTo(v.Type()).Implements(typeBinaryUnmarshaler) {
if v.CanAddr() {
v = v.Addr()
if u, ok := v.Interface().(encoding.BinaryUnmarshaler); ok {
@ -3172,7 +3225,7 @@ func isHashableValue(rv reflect.Value) bool {
// This function also handles nested tags.
// CBOR data is already verified to be well-formed before this function is used,
// so the recursion won't exceed max nested levels.
func convertByteSliceToByteString(v interface{}) (interface{}, bool) {
func convertByteSliceToByteString(v any) (any, bool) {
switch v := v.(type) {
case []byte:
return ByteString(v), true

File diff suppressed because it is too large Load Diff

51
doc.go
View File

@ -2,15 +2,15 @@
// Licensed under the MIT License. See LICENSE in the project root for license information.
/*
Package cbor is a modern CBOR codec (RFC 8949 & RFC 7049) with CBOR tags,
Go struct tags (toarray/keyasint/omitempty), Core Deterministic Encoding,
Package cbor is a modern CBOR codec (RFC 8949 & RFC 8742) with CBOR tags,
Go struct tag options (toarray/keyasint/omitempty/omitzero), Core Deterministic Encoding,
CTAP2, Canonical CBOR, float64->32->16, and duplicate map key detection.
Encoding options allow "preferred serialization" by encoding integers and floats
to their smallest forms (e.g. float16) when values fit.
Struct tags like "keyasint", "toarray" and "omitempty" make CBOR data smaller
and easier to use with structs.
Struct tag options "keyasint", "toarray", "omitempty", and "omitzero" reduce encoding size
and reduce programming effort.
For example, "toarray" tag makes struct fields encode to CBOR array elements. And
"keyasint" makes a field encode to an element of CBOR map with specified int key.
@ -23,11 +23,19 @@ The Quick Start guide is at https://github.com/fxamacker/cbor#quick-start
Function signatures identical to encoding/json include:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode.
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode
Standard interfaces include:
BinaryMarshaler, BinaryUnmarshaler, Marshaler, and Unmarshaler.
BinaryMarshaler, BinaryUnmarshaler, Marshaler, and Unmarshaler
Diagnostic functions translate CBOR data item into Diagnostic Notation:
Diagnose, DiagnoseFirst
Functions that simplify using CBOR Sequences (RFC 8742) include:
UnmarshalFirst
Custom encoding and decoding is possible by implementing standard interfaces for
user-defined Go types.
@ -50,19 +58,19 @@ Modes are intended to be reused and are safe for concurrent use.
EncMode and DecMode Interfaces
// EncMode interface uses immutable options and is safe for concurrent use.
type EncMode interface {
// EncMode interface uses immutable options and is safe for concurrent use.
type EncMode interface {
Marshal(v interface{}) ([]byte, error)
NewEncoder(w io.Writer) *Encoder
EncOptions() EncOptions // returns copy of options
}
}
// DecMode interface uses immutable options and is safe for concurrent use.
type DecMode interface {
// DecMode interface uses immutable options and is safe for concurrent use.
type DecMode interface {
Unmarshal(data []byte, v interface{}) error
NewDecoder(r io.Reader) *Decoder
DecOptions() DecOptions // returns copy of options
}
}
Using Default Encoding Mode
@ -78,6 +86,16 @@ Using Default Decoding Mode
decoder := cbor.NewDecoder(r)
err = decoder.Decode(&v)
Using Default Mode of UnmarshalFirst to Decode CBOR Sequences
// Decode the first CBOR data item and return remaining bytes:
rest, err = cbor.UnmarshalFirst(b, &v) // decode []byte b to v
Using Extended Diagnostic Notation (EDN) to represent CBOR data
// Translate the first CBOR data item into text and return remaining bytes.
text, rest, err = cbor.DiagnoseFirst(b) // decode []byte b to text
Creating and Using Encoding Modes
// Create EncOptions using either struct literal or a function.
@ -111,15 +129,20 @@ Decoding Options: https://github.com/fxamacker/cbor#decoding-options
Struct tags like `cbor:"name,omitempty"` and `json:"name,omitempty"` work as expected.
If both struct tags are specified then `cbor` is used.
Struct tags like "keyasint", "toarray", and "omitempty" make it easy to use
Struct tag options like "keyasint", "toarray", "omitempty", and "omitzero" make it easy to use
very compact formats like COSE and CWT (CBOR Web Tokens) with structs.
The "omitzero" option omits zero values from encoding, matching
[stdlib encoding/json behavior](https://pkg.go.dev/encoding/json#Marshal).
When specified in the `cbor` tag, the option is always honored.
When specified in the `json` tag, the option is honored when building with Go 1.24+.
For example, "toarray" makes struct fields encode to array elements. And "keyasint"
makes struct fields encode to elements of CBOR map with int keys.
https://raw.githubusercontent.com/fxamacker/images/master/cbor/v2.0.0/cbor_easy_api.png
Struct tags are listed at https://github.com/fxamacker/cbor#struct-tags-1
Struct tag options are listed at https://github.com/fxamacker/cbor#struct-tags-1
# Tests and Fuzzing

202
encode.go
View File

@ -58,8 +58,10 @@ import (
//
// Marshal supports format string stored under the "cbor" key in the struct
// field's tag. CBOR format string can specify the name of the field,
// "omitempty" and "keyasint" options, and special case "-" for field omission.
// If "cbor" key is absent, Marshal uses "json" key.
// "omitempty", "omitzero" and "keyasint" options, and special case "-" for
// field omission. If "cbor" key is absent, Marshal uses "json" key.
// When using the "json" key, the "omitzero" option is honored when building
// with Go 1.24+ to match stdlib encoding/json behavior.
//
// Struct field name is treated as integer if it has "keyasint" option in
// its format string. The format string must specify an integer as its
@ -67,8 +69,8 @@ import (
//
// Special struct field "_" is used to specify struct level options, such as
// "toarray". "toarray" option enables Go struct to be encoded as CBOR array.
// "omitempty" is disabled by "toarray" to ensure that the same number
// of elements are encoded every time.
// "omitempty" and "omitzero" are disabled by "toarray" to ensure that the
// same number of elements are encoded every time.
//
// Anonymous struct fields are marshaled as if their exported fields
// were fields in the outer struct. Marshal follows the same struct fields
@ -92,7 +94,7 @@ import (
//
// Values of other types cannot be encoded in CBOR. Attempting
// to encode such a value causes Marshal to return an UnsupportedTypeError.
func Marshal(v interface{}) ([]byte, error) {
func Marshal(v any) ([]byte, error) {
return defaultEncMode.Marshal(v)
}
@ -103,7 +105,7 @@ func Marshal(v interface{}) ([]byte, error) {
// partially encoded data if error is returned.
//
// See Marshal for more details.
func MarshalToBuffer(v interface{}, buf *bytes.Buffer) error {
func MarshalToBuffer(v any, buf *bytes.Buffer) error {
return defaultEncMode.MarshalToBuffer(v, buf)
}
@ -773,8 +775,9 @@ func (opts EncOptions) encMode() (*encMode, error) { //nolint:gocritic // ignore
// EncMode is the main interface for CBOR encoding.
type EncMode interface {
Marshal(v interface{}) ([]byte, error)
Marshal(v any) ([]byte, error)
NewEncoder(w io.Writer) *Encoder
NewStreamEncoder(w io.Writer) *StreamEncoder
EncOptions() EncOptions
}
@ -783,7 +786,7 @@ type EncMode interface {
// into the built-in buffer pool.
type UserBufferEncMode interface {
EncMode
MarshalToBuffer(v interface{}, buf *bytes.Buffer) error
MarshalToBuffer(v any, buf *bytes.Buffer) error
// This private method is to prevent users implementing
// this interface and so future additions to it will
@ -921,7 +924,7 @@ func (em *encMode) encTagBytes(t reflect.Type) []byte {
// Marshal returns the CBOR encoding of v using em encoding mode.
//
// See the documentation for Marshal for details.
func (em *encMode) Marshal(v interface{}) ([]byte, error) {
func (em *encMode) Marshal(v any) ([]byte, error) {
e := getEncodeBuffer()
if err := encode(e, em, reflect.ValueOf(v)); err != nil {
@ -943,7 +946,7 @@ func (em *encMode) Marshal(v interface{}) ([]byte, error) {
// partially encoded data if error is returned.
//
// See Marshal for more details.
func (em *encMode) MarshalToBuffer(v interface{}, buf *bytes.Buffer) error {
func (em *encMode) MarshalToBuffer(v any, buf *bytes.Buffer) error {
if buf == nil {
return fmt.Errorf("cbor: encoding buffer provided by user is nil")
}
@ -955,11 +958,19 @@ func (em *encMode) NewEncoder(w io.Writer) *Encoder {
return &Encoder{w: w, em: em}
}
// NewStreamEncoder returns a new stream encoder that writes sequentially to w using em EncMode.
func (em *encMode) NewStreamEncoder(w io.Writer) *StreamEncoder {
return &StreamEncoder{
Encoder: em.NewEncoder(w),
buf: getEncodeBuffer(),
}
}
// encodeBufferPool caches unused bytes.Buffer objects for later reuse.
var encodeBufferPool = sync.Pool{
New: func() interface{} {
New: func() any {
e := new(bytes.Buffer)
e.Grow(32) // TODO: make this configurable
e.Grow(64) // TODO: make this configurable
return e
},
}
@ -975,6 +986,7 @@ func putEncodeBuffer(e *bytes.Buffer) {
type encodeFunc func(e *bytes.Buffer, em *encMode, v reflect.Value) error
type isEmptyFunc func(em *encMode, v reflect.Value) (empty bool, err error)
type isZeroFunc func(v reflect.Value) (zero bool, err error)
func encode(e *bytes.Buffer, em *encMode, v reflect.Value) error {
if !v.IsValid() {
@ -983,7 +995,7 @@ func encode(e *bytes.Buffer, em *encMode, v reflect.Value) error {
return nil
}
vt := v.Type()
f, _ := getEncodeFunc(vt)
f, _, _ := getEncodeFunc(vt)
if f == nil {
return &UnsupportedTypeError{vt}
}
@ -1483,6 +1495,15 @@ func encodeStruct(e *bytes.Buffer, em *encMode, v reflect.Value) (err error) {
continue
}
}
if f.omitZero {
zero, err := f.izf(fv)
if err != nil {
return err
}
if zero {
continue
}
}
if !f.keyAsInt && em.fieldName == FieldNameToByteString {
e.Write(f.cborNameByteString)
@ -1775,34 +1796,34 @@ var (
typeByteString = reflect.TypeOf(ByteString(""))
)
func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc) {
func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc, izf isZeroFunc) {
k := t.Kind()
if k == reflect.Ptr {
return getEncodeIndirectValueFunc(t), isEmptyPtr
if k == reflect.Pointer {
return getEncodeIndirectValueFunc(t), isEmptyPtr, getIsZeroFunc(t)
}
switch t {
case typeSimpleValue:
return encodeMarshalerType, isEmptyUint
return encodeMarshalerType, isEmptyUint, getIsZeroFunc(t)
case typeTag:
return encodeTag, alwaysNotEmpty
return encodeTag, alwaysNotEmpty, getIsZeroFunc(t)
case typeTime:
return encodeTime, alwaysNotEmpty
return encodeTime, alwaysNotEmpty, getIsZeroFunc(t)
case typeBigInt:
return encodeBigInt, alwaysNotEmpty
return encodeBigInt, alwaysNotEmpty, getIsZeroFunc(t)
case typeRawMessage:
return encodeMarshalerType, isEmptySlice
return encodeMarshalerType, isEmptySlice, getIsZeroFunc(t)
case typeByteString:
return encodeMarshalerType, isEmptyString
return encodeMarshalerType, isEmptyString, getIsZeroFunc(t)
}
if reflect.PtrTo(t).Implements(typeMarshaler) {
return encodeMarshalerType, alwaysNotEmpty
if reflect.PointerTo(t).Implements(typeMarshaler) {
return encodeMarshalerType, alwaysNotEmpty, getIsZeroFunc(t)
}
if reflect.PtrTo(t).Implements(typeBinaryMarshaler) {
if reflect.PointerTo(t).Implements(typeBinaryMarshaler) {
defer func() {
// capture encoding method used for modes that disable BinaryMarshaler
bme := binaryMarshalerEncoder{
@ -1815,39 +1836,39 @@ func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc) {
}
switch k {
case reflect.Bool:
return encodeBool, isEmptyBool
return encodeBool, isEmptyBool, getIsZeroFunc(t)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return encodeInt, isEmptyInt
return encodeInt, isEmptyInt, getIsZeroFunc(t)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return encodeUint, isEmptyUint
return encodeUint, isEmptyUint, getIsZeroFunc(t)
case reflect.Float32, reflect.Float64:
return encodeFloat, isEmptyFloat
return encodeFloat, isEmptyFloat, getIsZeroFunc(t)
case reflect.String:
return encodeString, isEmptyString
return encodeString, isEmptyString, getIsZeroFunc(t)
case reflect.Slice:
if t.Elem().Kind() == reflect.Uint8 {
return encodeByteString, isEmptySlice
return encodeByteString, isEmptySlice, getIsZeroFunc(t)
}
fallthrough
case reflect.Array:
f, _ := getEncodeFunc(t.Elem())
f, _, _ := getEncodeFunc(t.Elem())
if f == nil {
return nil, nil
return nil, nil, nil
}
return arrayEncodeFunc{f: f}.encode, isEmptySlice
return arrayEncodeFunc{f: f}.encode, isEmptySlice, getIsZeroFunc(t)
case reflect.Map:
f := getEncodeMapFunc(t)
if f == nil {
return nil, nil
return nil, nil, nil
}
return f, isEmptyMap
return f, isEmptyMap, getIsZeroFunc(t)
case reflect.Struct:
// Get struct's special field "_" tag options
@ -1855,31 +1876,31 @@ func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc) {
tag := f.Tag.Get("cbor")
if tag != "-" {
if hasToArrayOption(tag) {
return encodeStructToArray, isEmptyStruct
return encodeStructToArray, isEmptyStruct, isZeroFieldStruct
}
}
}
return encodeStruct, isEmptyStruct
return encodeStruct, isEmptyStruct, getIsZeroFunc(t)
case reflect.Interface:
return encodeIntf, isEmptyIntf
return encodeIntf, isEmptyIntf, getIsZeroFunc(t)
}
return nil, nil
return nil, nil, nil
}
func getEncodeIndirectValueFunc(t reflect.Type) encodeFunc {
for t.Kind() == reflect.Ptr {
for t.Kind() == reflect.Pointer {
t = t.Elem()
}
f, _ := getEncodeFunc(t)
f, _, _ := getEncodeFunc(t)
if f == nil {
return nil
}
return func(e *bytes.Buffer, em *encMode, v reflect.Value) error {
for v.Kind() == reflect.Ptr && !v.IsNil() {
for v.Kind() == reflect.Pointer && !v.IsNil() {
v = v.Elem()
}
if v.Kind() == reflect.Ptr && v.IsNil() {
if v.Kind() == reflect.Pointer && v.IsNil() {
e.Write(cborNil)
return nil
}
@ -1987,3 +2008,96 @@ func float32NaNFromReflectValue(v reflect.Value) float32 {
f32 := p.Convert(reflect.TypeOf((*float32)(nil))).Elem().Interface().(float32)
return f32
}
type isZeroer interface {
IsZero() bool
}
var isZeroerType = reflect.TypeOf((*isZeroer)(nil)).Elem()
// getIsZeroFunc returns a function for the given type that can be called to determine if a given value is zero.
// Types that implement `IsZero() bool` are delegated to for non-nil values.
// Types that do not implement `IsZero() bool` use the reflect.Value#IsZero() implementation.
// The returned function matches behavior of stdlib encoding/json behavior in Go 1.24+.
func getIsZeroFunc(t reflect.Type) isZeroFunc {
// Provide a function that uses a type's IsZero method if defined.
switch {
case t == nil:
return isZeroDefault
case t.Kind() == reflect.Interface && t.Implements(isZeroerType):
return isZeroInterfaceCustom
case t.Kind() == reflect.Pointer && t.Implements(isZeroerType):
return isZeroPointerCustom
case t.Implements(isZeroerType):
return isZeroCustom
case reflect.PointerTo(t).Implements(isZeroerType):
return isZeroAddrCustom
default:
return isZeroDefault
}
}
// isZeroInterfaceCustom returns true for nil or pointer-to-nil values,
// and delegates to the custom IsZero() implementation otherwise.
func isZeroInterfaceCustom(v reflect.Value) (bool, error) {
kind := v.Kind()
switch kind {
case reflect.Chan, reflect.Func, reflect.Map, reflect.Pointer, reflect.Interface, reflect.Slice:
if v.IsNil() {
return true, nil
}
}
switch kind {
case reflect.Interface, reflect.Pointer:
if elem := v.Elem(); elem.Kind() == reflect.Pointer && elem.IsNil() {
return true, nil
}
}
return v.Interface().(isZeroer).IsZero(), nil
}
// isZeroPointerCustom returns true for nil values,
// and delegates to the custom IsZero() implementation otherwise.
func isZeroPointerCustom(v reflect.Value) (bool, error) {
if v.IsNil() {
return true, nil
}
return v.Interface().(isZeroer).IsZero(), nil
}
// isZeroCustom delegates to the custom IsZero() implementation.
func isZeroCustom(v reflect.Value) (bool, error) {
return v.Interface().(isZeroer).IsZero(), nil
}
// isZeroAddrCustom delegates to the custom IsZero() implementation of the addr of the value.
func isZeroAddrCustom(v reflect.Value) (bool, error) {
if !v.CanAddr() {
// Temporarily box v so we can take the address.
v2 := reflect.New(v.Type()).Elem()
v2.Set(v)
v = v2
}
return v.Addr().Interface().(isZeroer).IsZero(), nil
}
// isZeroDefault calls reflect.Value#IsZero()
func isZeroDefault(v reflect.Value) (bool, error) {
if !v.IsValid() {
// v is zero value
return true, nil
}
return v.IsZero(), nil
}
// isZeroFieldStruct is used to determine whether to omit toarray structs
func isZeroFieldStruct(v reflect.Value) (bool, error) {
structType, err := getEncodingStructType(v.Type())
if err != nil {
return false, err
}
return len(structType.fields) == 0, nil
}

View File

@ -1,8 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build go1.20
package cbor
import (
@ -67,8 +65,8 @@ func (me *mapKeyValueEncodeFunc) encodeKeyValues(e *bytes.Buffer, em *encMode, v
}
func getEncodeMapFunc(t reflect.Type) encodeFunc {
kf, _ := getEncodeFunc(t.Key())
ef, _ := getEncodeFunc(t.Elem())
kf, _, _ := getEncodeFunc(t.Key())
ef, _, _ := getEncodeFunc(t.Elem())
if kf == nil || ef == nil {
return nil
}
@ -76,13 +74,13 @@ func getEncodeMapFunc(t reflect.Type) encodeFunc {
kf: kf,
ef: ef,
kpool: sync.Pool{
New: func() interface{} {
New: func() any {
rk := reflect.New(t.Key()).Elem()
return &rk
},
},
vpool: sync.Pool{
New: func() interface{} {
New: func() any {
rv := reflect.New(t.Elem()).Elem()
return &rv
},

View File

@ -1,60 +0,0 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build !go1.20
package cbor
import (
"bytes"
"reflect"
)
type mapKeyValueEncodeFunc struct {
kf, ef encodeFunc
}
func (me *mapKeyValueEncodeFunc) encodeKeyValues(e *bytes.Buffer, em *encMode, v reflect.Value, kvs []keyValue) error {
if kvs == nil {
for i, iter := 0, v.MapRange(); iter.Next(); i++ {
if err := me.kf(e, em, iter.Key()); err != nil {
return err
}
if err := me.ef(e, em, iter.Value()); err != nil {
return err
}
}
return nil
}
initial := e.Len()
for i, iter := 0, v.MapRange(); iter.Next(); i++ {
offset := e.Len()
if err := me.kf(e, em, iter.Key()); err != nil {
return err
}
valueOffset := e.Len()
if err := me.ef(e, em, iter.Value()); err != nil {
return err
}
kvs[i] = keyValue{
offset: offset - initial,
valueOffset: valueOffset - initial,
nextOffset: e.Len() - initial,
}
}
return nil
}
func getEncodeMapFunc(t reflect.Type) encodeFunc {
kf, _ := getEncodeFunc(t.Key())
ef, _ := getEncodeFunc(t.Elem())
if kf == nil || ef == nil {
return nil
}
mkv := &mapKeyValueEncodeFunc{kf: kf, ef: ef}
return mapEncodeFunc{
e: mkv.encodeKeyValues,
}.encode
}

File diff suppressed because it is too large Load Diff

View File

@ -82,7 +82,7 @@ func ExampleMarshal_canonical() {
// a46341676504644d616c65f4644e616d656543616e647968436f6e7461637473a2634a6f656c3232322d3232322d32323232644d6172796c3131312d3131312d31313131
}
// This example uses "toarray" struct tag to encode struct as CBOR array.
// This example uses "toarray" struct tag option to encode struct as CBOR array.
func ExampleMarshal_toarray() {
type Record struct {
_ struct{} `cbor:",toarray"`
@ -100,7 +100,7 @@ func ExampleMarshal_toarray() {
// 836763757272656e74615601
}
// This example uses "keyasint" struct tag to encode struct's fiele names as integer.
// This example uses "keyasint" struct tag option to encode struct's field names as integer.
// This feautre is very useful in handling COSE, CWT, SenML data.
func ExampleMarshal_keyasint() {
type Record struct {
@ -353,7 +353,7 @@ func ExampleDecoder() {
}
func Example_cWT() {
// Use "keyasint" struct tag to encode/decode struct to/from CBOR map.
// Use "keyasint" struct tag option to encode/decode struct to/from CBOR map.
type claims struct {
Iss string `cbor:"1,keyasint"`
Sub string `cbor:"2,keyasint"`
@ -403,14 +403,14 @@ func Example_cWTWithDupMapKeyOption() {
}
func Example_signedCWT() {
// Use "keyasint" struct tag to encode/decode struct to/from CBOR map.
// Use "keyasint" struct tag option to encode/decode struct to/from CBOR map.
// Partial COSE header definition
type coseHeader struct {
Alg int `cbor:"1,keyasint,omitempty"`
Kid []byte `cbor:"4,keyasint,omitempty"`
IV []byte `cbor:"5,keyasint,omitempty"`
}
// Use "toarray" struct tag to encode/decode struct to/from CBOR array.
// Use "toarray" struct tag option to encode/decode struct to/from CBOR array.
type signedCWT struct {
_ struct{} `cbor:",toarray"`
Protected []byte
@ -433,14 +433,14 @@ func Example_signedCWT() {
}
func Example_signedCWTWithTag() {
// Use "keyasint" struct tag to encode/decode struct to/from CBOR map.
// Use "keyasint" struct tag option to encode/decode struct to/from CBOR map.
// Partial COSE header definition
type coseHeader struct {
Alg int `cbor:"1,keyasint,omitempty"`
Kid []byte `cbor:"4,keyasint,omitempty"`
IV []byte `cbor:"5,keyasint,omitempty"`
}
// Use "toarray" struct tag to encode/decode struct to/from CBOR array.
// Use "toarray" struct tag option to encode/decode struct to/from CBOR array.
type signedCWT struct {
_ struct{} `cbor:",toarray"`
Protected []byte
@ -478,7 +478,7 @@ func Example_signedCWTWithTag() {
}
func Example_cOSE() {
// Use "keyasint" struct tag to encode/decode struct to/from CBOR map.
// Use "keyasint" struct tag option to encode/decode struct to/from CBOR map.
// Use cbor.RawMessage to delay unmarshaling (CrvOrNOrK's data type depends on Kty's value).
type coseKey struct {
Kty int `cbor:"1,keyasint,omitempty"`
@ -507,7 +507,7 @@ func Example_cOSE() {
}
func Example_senML() {
// Use "keyasint" struct tag to encode/decode struct to/from CBOR map.
// Use "keyasint" struct tag option to encode/decode struct to/from CBOR map.
type SenMLRecord struct {
BaseName string `cbor:"-2,keyasint,omitempty"`
BaseTime float64 `cbor:"-3,keyasint,omitempty"`

2
go.mod
View File

@ -1,5 +1,5 @@
module github.com/fxamacker/cbor/v2
go 1.17 // Compiling with go 1.20+ uses new features and optimizations
go 1.20
require github.com/x448/float16 v0.8.4

View File

@ -1,3 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor_test
import (
@ -34,7 +37,7 @@ func TestStdlibJSONCompatibility(t *testing.T) {
for _, tc := range []struct {
name string
original interface{}
original any
ifaceEqual bool // require equal intermediate interface{} values from both protocols
}{
{
@ -67,14 +70,14 @@ func TestStdlibJSONCompatibility(t *testing.T) {
}
t.Logf("original to cbor: %s", diag1)
var jintf interface{}
var jintf any
err = json.Unmarshal(j1, &jintf)
if err != nil {
t.Fatal(err)
}
t.Logf("json to interface{} (%T): %#v", jintf, jintf)
var cintf interface{}
var cintf any
err = dec.Unmarshal(c1, &cintf)
if err != nil {
t.Fatal(err)

8
omitzero_go124.go Normal file
View File

@ -0,0 +1,8 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build go1.24
package cbor
var jsonStdlibSupportsOmitzero = true

8
omitzero_pre_go124.go Normal file
View File

@ -0,0 +1,8 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build !go1.24
package cbor
var jsonStdlibSupportsOmitzero = false

View File

@ -1,3 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor
import (
@ -45,6 +48,9 @@ func (sv SimpleValue) MarshalCBOR() ([]byte, error) {
}
// UnmarshalCBOR decodes CBOR simple value (major type 7) to SimpleValue.
//
// Deprecated: No longer used by this codec; kept for compatibility
// with user apps that directly call this function.
func (sv *SimpleValue) UnmarshalCBOR(data []byte) error {
if sv == nil {
return errors.New("cbor.SimpleValue: UnmarshalCBOR on nil pointer")
@ -52,6 +58,29 @@ func (sv *SimpleValue) UnmarshalCBOR(data []byte) error {
d := decoder{data: data, dm: defaultDecMode}
// Check well-formedness of CBOR data item.
// SimpleValue.UnmarshalCBOR() is exported, so
// the codec needs to support same behavior for:
// - Unmarshal(data, *SimpleValue)
// - SimpleValue.UnmarshalCBOR(data)
err := d.wellformed(false, false)
if err != nil {
return err
}
return sv.unmarshalCBOR(data)
}
// unmarshalCBOR decodes CBOR simple value (major type 7) to SimpleValue.
// This function assumes data is well-formed, and does not perform bounds checking.
// This function is called by Unmarshal().
func (sv *SimpleValue) unmarshalCBOR(data []byte) error {
if sv == nil {
return errors.New("cbor.SimpleValue: UnmarshalCBOR on nil pointer")
}
d := decoder{data: data, dm: defaultDecMode}
typ, ai, val := d.getHead()
if typ != cborTypePrimitives {

View File

@ -5,7 +5,9 @@ package cbor
import (
"bytes"
"io"
"reflect"
"strings"
"testing"
)
@ -51,8 +53,127 @@ func TestUnmarshalSimpleValue(t *testing.T) {
})
}
func TestUnmarshalSimpleValueOnBadData(t *testing.T) {
testCases := []struct {
name string
data []byte
errMsg string
}{
// Empty data
{
name: "nil data",
data: nil,
errMsg: io.EOF.Error(),
},
{
name: "empty data",
data: []byte{},
errMsg: io.EOF.Error(),
},
// Wrong CBOR types
{
name: "uint type",
data: hexDecode("01"),
errMsg: "cbor: cannot unmarshal positive integer into Go value of type SimpleValue",
},
{
name: "int type",
data: hexDecode("20"),
errMsg: "cbor: cannot unmarshal negative integer into Go value of type SimpleValue",
},
{
name: "byte string type",
data: hexDecode("40"),
errMsg: "cbor: cannot unmarshal byte string into Go value of type SimpleValue",
},
{
name: "string type",
data: hexDecode("60"),
errMsg: "cbor: cannot unmarshal UTF-8 text string into Go value of type SimpleValue",
},
{
name: "array type",
data: hexDecode("80"),
errMsg: "cbor: cannot unmarshal array into Go value of type SimpleValue",
},
{
name: "map type",
data: hexDecode("a0"),
errMsg: "cbor: cannot unmarshal map into Go value of type SimpleValue",
},
{
name: "tag type",
data: hexDecode("c074323031332d30332d32315432303a30343a30305a"),
errMsg: "cbor: cannot unmarshal tag into Go value of type SimpleValue",
},
{
name: "float type",
data: hexDecode("f90000"),
errMsg: "cbor: cannot unmarshal primitives into Go value of type SimpleValue",
},
// Truncated CBOR data
{
name: "truncated head",
data: hexDecode("18"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
// Truncated CBOR simple value
{
name: "truncated simple value",
data: hexDecode("f8"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
// Invalid simple value
{
name: "invalid simple value",
data: hexDecode("f800"),
errMsg: "cbor: invalid simple value 0 for type primitives",
},
// Extraneous CBOR data
{
name: "extraneous data",
data: hexDecode("f4f5"),
errMsg: "cbor: 1 bytes of extraneous data starting at index 1",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Test SimpleValue.UnmarshalCBOR(data)
{
var v SimpleValue
err := v.UnmarshalCBOR(tc.data)
if err == nil {
t.Errorf("UnmarshalCBOR(%x) didn't return error", tc.data)
}
if !strings.HasPrefix(err.Error(), tc.errMsg) {
t.Errorf("UnmarshalCBOR(%x) returned error %q, want %q", tc.data, err.Error(), tc.errMsg)
}
}
// Test Unmarshal(data, *SimpleValue), which calls SimpleValue.unmarshalCBOR() under the hood
{
var v SimpleValue
err := Unmarshal(tc.data, &v)
if err == nil {
t.Errorf("Unmarshal(%x) didn't return error", tc.data)
}
if !strings.HasPrefix(err.Error(), tc.errMsg) {
t.Errorf("Unmarshal(%x) returned error %q, want %q", tc.data, err.Error(), tc.errMsg)
}
}
})
}
}
func testUnmarshalInvalidSimpleValueToEmptyInterface(t *testing.T, data []byte) {
var v interface{}
var v any
if err := Unmarshal(data, v); err == nil {
t.Errorf("Unmarshal(0x%x) didn't return an error", data)
} else if _, ok := err.(*SyntaxError); !ok {
@ -69,8 +190,8 @@ func testUnmarshalInvalidSimpleValue(t *testing.T, data []byte) {
}
}
func testUnmarshalSimpleValueToEmptyInterface(t *testing.T, data []byte, want interface{}) {
var v interface{}
func testUnmarshalSimpleValueToEmptyInterface(t *testing.T, data []byte, want any) {
var v any
if err := Unmarshal(data, &v); err != nil {
t.Errorf("Unmarshal(0x%x) returned error %v", data, err)
return

View File

@ -13,7 +13,7 @@ import (
// Decoder reads and decodes CBOR values from io.Reader.
type Decoder struct {
r io.Reader
d decoder
d *decoder
buf []byte
off int // next read offset in buf
bytesRead int
@ -26,7 +26,7 @@ func NewDecoder(r io.Reader) *Decoder {
}
// Decode reads CBOR value and decodes it into the value pointed to by v.
func (dec *Decoder) Decode(v interface{}) error {
func (dec *Decoder) Decode(v any) error {
_, err := dec.readNext()
if err != nil {
// Return validation error or read error.
@ -132,8 +132,14 @@ func (dec *Decoder) readNext() (int, error) {
// - dec.buf contains previously unread data and new data.
// - dec.off is 0.
func (dec *Decoder) read() (int, error) {
// Grow buf if needed.
if dec.r == nil {
// buf contains all the data, can't read more.
return 0, io.EOF
}
const minRead = 512
// Grow buf if needed.
if cap(dec.buf)-len(dec.buf)+dec.off < minRead {
oldUnreadBuf := dec.buf[dec.off:]
dec.buf = make([]byte, len(dec.buf)-dec.off, 2*cap(dec.buf)+minRead)
@ -169,22 +175,38 @@ func NewEncoder(w io.Writer) *Encoder {
return defaultEncMode.NewEncoder(w)
}
func (enc *Encoder) checkIndefiniteLengthDataItemTypeIfNeeded(v interface{}) error {
if len(enc.indefTypes) == 0 || v == nil {
return nil
}
indefType := enc.indefTypes[len(enc.indefTypes)-1]
return checkIndefiniteLengthDataItemType(indefType, v)
}
func checkIndefiniteLengthDataItemType(indefType cborType, v interface{}) error {
switch indefType {
case cborTypeTextString:
k := reflect.TypeOf(v).Kind()
if k != reflect.String {
return errors.New("cbor: cannot encode item type " + k.String() + " for indefinite-length text string")
}
case cborTypeByteString:
t := reflect.TypeOf(v)
k := t.Kind()
if (k != reflect.Array && k != reflect.Slice) || t.Elem().Kind() != reflect.Uint8 {
return errors.New("cbor: cannot encode item type " + k.String() + " for indefinite-length byte string")
}
}
return nil
}
// Encode writes the CBOR encoding of v.
func (enc *Encoder) Encode(v interface{}) error {
if len(enc.indefTypes) > 0 && v != nil {
indefType := enc.indefTypes[len(enc.indefTypes)-1]
if indefType == cborTypeTextString {
k := reflect.TypeOf(v).Kind()
if k != reflect.String {
return errors.New("cbor: cannot encode item type " + k.String() + " for indefinite-length text string")
}
} else if indefType == cborTypeByteString {
t := reflect.TypeOf(v)
k := t.Kind()
if (k != reflect.Array && k != reflect.Slice) || t.Elem().Kind() != reflect.Uint8 {
return errors.New("cbor: cannot encode item type " + k.String() + " for indefinite-length byte string")
}
}
if err := enc.checkIndefiniteLengthDataItemTypeIfNeeded(v); err != nil {
return err
}
buf := getEncodeBuffer()

641
stream_decode.go Normal file
View File

@ -0,0 +1,641 @@
package cbor
import (
"encoding/binary"
"errors"
"fmt"
"io"
"math"
"math/big"
"unicode/utf8"
)
// Type represents CBOR type.
type Type uint8
const (
UndefinedType Type = iota
// CBOR major types
UintType // CBOR major type 0
IntType // CBOR major type 1
ByteStringType // CBOR major type 2
TextStringType // CBOR major type 3
ArrayType // CBOR major type 4
MapType // CBOR major type 5
TagType // CBOR major type 6
// OtherType is CBOR major type 7. It is for two types of data:
// floating-point numbers and "simple values" that do not need any content.
OtherType
// Non-major types
BigNumType // BigNumType is specified as part of CBOR major type 6
NilType // NilType is specified as part of CBOR major type 7
BoolType // BoolType is specified as part of CBOR major type 7
)
func (t Type) String() string {
switch t {
case UintType:
return "CBOR uint type"
case IntType:
return "CBOR int type"
case ByteStringType:
return "CBOR byte string type"
case TextStringType:
return "CBOR text string type"
case ArrayType:
return "CBOR array type"
case MapType:
return "CBOR map type"
case TagType:
return "CBOR tag type"
case OtherType:
return "CBOR other type"
case NilType:
return "CBOR nil type"
case BoolType:
return "CBOR boolean type"
case BigNumType:
return "CBOR bignum type"
default:
return "undefined CBOR type"
}
}
type WrongTypeError struct {
ActualType Type
ExpectedType string
}
func (e *WrongTypeError) Error() string {
return fmt.Sprintf("cbor: cannot decode %s to %s", e.ActualType.String(), e.ExpectedType)
}
// IndefiniteLengthNotSupportedError is returned by NextSize() if next CBOR data
// is indefinite length.
type IndefiniteLengthNotSupportedError struct {
t Type
}
func (e *IndefiniteLengthNotSupportedError) Error() string {
return fmt.Sprintf("cbor: size is unavailable for indefinite length %s", e.t)
}
// TypeNotSupportedError is returned by NextSize() if next CBOR type
// is not supported/implemented by NextSize().
type TypeNotSupportedError struct {
t Type
}
func (e *TypeNotSupportedError) Error() string {
return fmt.Sprintf("cbor: size operation is not supported for %s", e.t)
}
// StreamDecoder validates complete CBOR data and decodes it in chunks.
//
// If CBOR data is malformed or fails decoding checks (set by options),
// validation error is saved and all subseqent decoding functions
// return the saved error.
//
// If DecodeXXX() tries to decode CBOR data of mismatched type,
// WrongTypeError is returned. Caller can retry to decode the same
// data with a different DecodeXXX().
//
// If DecodeXXX() function returns other types of error,
// CBOR data is skipped. User can decode next CBOR data.
type StreamDecoder struct {
dec *Decoder
err error
remainingBytes int // remaining bytes of a complete and validated CBOR data
decodedMsgBytes int // number of bytes of decoded CBOR messages
}
// NewStreamDecoder returns a new StreamDecoder that reads from r using default DecMode.
func NewStreamDecoder(r io.Reader) *StreamDecoder {
return defaultDecMode.NewStreamDecoder(r)
}
// NewByteStreamDecoder returns a new StreamDecoder that reads from data using default DecMode.
func NewByteStreamDecoder(data []byte) *StreamDecoder {
return defaultDecMode.NewByteStreamDecoder(data)
}
// NextType returns the next CBOR type.
func (sd *StreamDecoder) NextType() (Type, error) {
if err := sd.prepareNext(); err != nil {
return UndefinedType, err
}
b := sd.dec.d.data[sd.dec.d.off]
switch b & 0xe0 {
case 0x00:
return UintType, nil
case 0x20:
return IntType, nil
case 0x40:
return ByteStringType, nil
case 0x60:
return TextStringType, nil
case 0x80:
return ArrayType, nil
case 0xa0:
return MapType, nil
case 0xc0:
if b == 0xc2 || b == 0xc3 {
return BigNumType, nil
}
return TagType, nil
case 0xe0:
switch b {
case 0xf4, 0xf5:
return BoolType, nil
case 0xf6:
return NilType, nil
}
return OtherType, nil
}
return UndefinedType, errors.New("cbor: unrecognized type")
}
// NextSize returns the next CBOR data size for five types.
// Returned uint64 represents different kind of size depending on
// the type:
// - ByteStringType : length (in bytes) of byte string
// - TextStringType : length (in bytes) of text string
// - ArrayType : number of array elements
// - MapType : number of key/value pairs
// - BigNumType : length (in bytes) of encoded big number
// Error is returned for indef length data and unsupported types.
// Support for additional types wasn't added for simplicity.
func (sd *StreamDecoder) NextSize() (uint64, error) {
t, err := sd.NextType()
if err != nil {
return 0, err
}
off := sd.dec.d.off
if t == BigNumType {
// NextType() validates CBOR tag data (tag number + tag content),
// so it's safe to increment offset to access tag content.
off++ // Increment offset to point to tag content
t = ByteStringType // tag content type is CBOR byte string
}
switch t {
case ByteStringType, TextStringType, ArrayType, MapType:
ai := sd.dec.d.data[off] & 0x1f
off++
// This function only interprets valid head because
// CBOR data is validated in NextType().
var size uint64
if ai < 24 {
size = uint64(ai)
} else if ai == 24 {
size = uint64(sd.dec.d.data[off])
} else if ai == 25 {
size = uint64(binary.BigEndian.Uint16(sd.dec.d.data[off : off+2]))
} else if ai == 26 {
size = uint64(binary.BigEndian.Uint32(sd.dec.d.data[off : off+4]))
} else if ai == 27 {
size = binary.BigEndian.Uint64(sd.dec.d.data[off : off+8])
} else if ai == 31 {
return 0, &IndefiniteLengthNotSupportedError{t}
}
return size, nil
}
return 0, &TypeNotSupportedError{t}
}
// Skip skips next CBOR data.
func (sd *StreamDecoder) Skip() error {
if err := sd.prepareNext(); err != nil {
return err
}
d := sd.dec.d
start := d.off
d.skip()
end := d.off
sd.updateState(end - start)
return nil
}
// DecodeRawBytes returns a copy of next CBOR data as raw bytes.
func (sd *StreamDecoder) DecodeRawBytes() ([]byte, error) {
if err := sd.prepareNext(); err != nil {
return nil, err
}
d := sd.dec.d
start := d.off
d.skip()
end := d.off
b := make([]byte, end-start)
copy(b, d.data[start:end])
sd.updateState(end - start)
return b, nil
}
// DecodeRawBytesZeroCopy returns next CBOR data as raw bytes pointing to underlying data.
// It is only available for StreamDecoder created with NewByteStreamDecoder.
func (sd *StreamDecoder) DecodeRawBytesZeroCopy() ([]byte, error) {
if sd.dec.r != nil {
return nil, errors.New("cbor: DecodeRawBytesZeroCopy is only supported for StreamDecoder created with NewByteStreamDecoder")
}
if err := sd.prepareNext(); err != nil {
return nil, err
}
d := sd.dec.d
start := d.off
d.skip()
end := d.off
b := d.data[start:end]
sd.updateState(end - start)
return b, nil
}
// DecodeNil decodes next CBOR data as nil. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeNil() error {
if err := sd.prepareNext(); err != nil {
return err
}
d := sd.dec.d
if d.data[d.off] == 0xf6 {
d.off++
sd.updateState(1)
return nil
}
t, _ := sd.NextType()
return &WrongTypeError{t, "nil"}
}
// DecodeBool decodes next CBOR data as bool. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeBool() (bool, error) {
if err := sd.prepareNext(); err != nil {
return false, err
}
d := sd.dec.d
b := d.data[d.off]
switch b {
case 0xf4, 0xf5:
d.off++
sd.updateState(1)
return b == 0xf5, nil
default:
t, _ := sd.NextType()
return false, &WrongTypeError{t, "bool"}
}
}
// DecodeUint64 decodes next CBOR data as uint64. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeUint64() (uint64, error) {
if err := sd.prepareNext(); err != nil {
return 0, err
}
d := sd.dec.d
if d.nextCBORType() != cborTypePositiveInt {
t, _ := sd.NextType()
return 0, &WrongTypeError{t, "uint64"}
}
start := d.off
_, _, val := d.getHead()
end := d.off
sd.updateState(end - start)
return val, nil
}
// DecodeInt64 decodes next CBOR data as int64. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeInt64() (int64, error) {
if err := sd.prepareNext(); err != nil {
return 0, err
}
d := sd.dec.d
switch nt := d.nextCBORType(); nt {
case cborTypePositiveInt, cborTypeNegativeInt:
start := d.off
_, _, val := d.getHead()
end := d.off
sd.updateState(end - start)
if val > math.MaxInt64 {
return 0, fmt.Errorf("cbor: %d overflow Go's int64", val)
}
if nt == cborTypePositiveInt {
return int64(val), nil
}
nValue := int64(-1) ^ int64(val)
return nValue, nil
default:
t, _ := sd.NextType()
return 0, &WrongTypeError{t, "int64"}
}
}
// DecodeBytes decodes next CBOR data as []byte. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeBytes() ([]byte, error) {
if err := sd.prepareNext(); err != nil {
return nil, err
}
d := sd.dec.d
if d.nextCBORType() != cborTypeByteString {
t, _ := sd.NextType()
return nil, &WrongTypeError{t, "bytes"}
}
start := d.off
_, ai, val := d.getHead()
if ai == 31 {
// Indefinite length byte string isn't supported in StreamDecoder. Skip it.
d.off = start
_ = sd.Skip()
return nil, errors.New("cbor: indefinite length byte string isn't supported")
}
b := make([]byte, int(val))
copy(b, d.data[d.off:d.off+int(val)])
d.off += int(val)
end := d.off
sd.updateState(end - start)
return b, nil
}
// DecodeString decodes next CBOR data as string. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeString() (string, error) {
if err := sd.prepareNext(); err != nil {
return "", err
}
d := sd.dec.d
if d.nextCBORType() != cborTypeTextString {
t, _ := sd.NextType()
return "", &WrongTypeError{t, "string"}
}
start := d.off
_, ai, val := d.getHead()
if ai == 31 {
// Indefinite length text string isn't supported in StreamDecoder. Skip it.
d.off = start
_ = sd.Skip()
return "", errors.New("cbor: indefinite length text string isn't supported")
}
b := d.data[d.off : d.off+int(val)]
d.off += int(val)
end := d.off
sd.updateState(end - start)
if d.dm.utf8 == UTF8RejectInvalid && !utf8.Valid(b) {
return "", &SemanticError{"cbor: invalid UTF-8 string"}
}
return string(b), nil
}
// DecodeBigInt decodes next CBOR data as *big.Int. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeBigInt() (*big.Int, error) {
if err := sd.prepareNext(); err != nil {
return nil, err
}
d := sd.dec.d
t := d.data[d.off]
if t != 0xc2 && t != 0xc3 {
t, _ := sd.NextType()
return nil, &WrongTypeError{t, "big.Int"}
}
d.off++
sd.updateState(1)
b, err := sd.DecodeBytes()
if err != nil {
return nil, err
}
bi := new(big.Int).SetBytes(b)
if t == 0xc3 {
bi.Add(bi, bigOne)
bi.Neg(bi)
}
return bi, nil
}
// DecodeTagNumber decodes next CBOR data as tag number. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeTagNumber() (uint64, error) {
if err := sd.prepareNext(); err != nil {
return 0, err
}
d := sd.dec.d
if d.nextCBORType() != cborTypeTag {
t, _ := sd.NextType()
return 0, &WrongTypeError{t, "tag"}
}
start := d.off
_, _, val := d.getHead()
end := d.off
sd.updateState(end - start)
return val, nil
}
// DecodeArrayHead decodes next CBOR data as array size. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeArrayHead() (uint64, error) {
if err := sd.prepareNext(); err != nil {
return 0, err
}
d := sd.dec.d
if d.nextCBORType() != cborTypeArray {
t, _ := sd.NextType()
return 0, &WrongTypeError{t, "array"}
}
start := d.off
_, ai, val := d.getHead()
end := d.off
if ai == 31 {
// Indefinite length array isn't supported in StreamDecoder. Skip it.
d.off = start
_ = sd.Skip()
return 0, errors.New("cbor: indefinite length array isn't supported")
}
sd.updateState(end - start)
return val, nil
}
// DecodeMapHead decodes next CBOR data as map size. WrongType error is returned if type is mismatched.
func (sd *StreamDecoder) DecodeMapHead() (uint64, error) {
if err := sd.prepareNext(); err != nil {
return 0, err
}
d := sd.dec.d
if d.nextCBORType() != cborTypeMap {
t, _ := sd.NextType()
return 0, &WrongTypeError{t, "map"}
}
start := d.off
_, ai, val := d.getHead()
end := d.off
if ai == 31 {
// Indefinite length map isn't supported in StreamDecoder. Skip it.
d.off = start
_ = sd.Skip()
return 0, errors.New("cbor: indefinite length map isn't supported")
}
sd.updateState(end - start)
return val, nil
}
// prepareNext reads and validates next CBOR data.
// prepareNext can return io error or CBOR validation error.
func (sd *StreamDecoder) prepareNext() error {
if sd.err != nil {
return sd.err
}
if sd.remainingBytes > 0 {
return nil
}
return sd.readAndValidateNext()
}
func (sd *StreamDecoder) readAndValidateNext() error {
lastMsgBytes := sd.dec.d.off
length, err := sd._readAndValidateNext()
if err != nil {
sd.err = err
return err
}
sd.decodedMsgBytes += lastMsgBytes
sd.remainingBytes = length
sd.dec.off += length
sd.dec.bytesRead += length
return nil
}
func (sd *StreamDecoder) _readAndValidateNext() (int, error) {
if len(sd.dec.buf) == sd.dec.off {
n, err := sd.dec.read()
if err != nil && err != io.EOF {
return 0, err
}
if n == 0 {
return 0, io.EOF
}
}
for {
sd.dec.d.reset(sd.dec.buf[sd.dec.off:])
start := sd.dec.d.off
err := sd.dec.d.wellformed(true, false)
end := sd.dec.d.off
// Restore decoder offset after validation
sd.dec.d.off = start
if err == nil {
// Next complete data is read and validated
return end - start, nil
}
if err != io.ErrUnexpectedEOF {
return 0, err
}
// valid() returned io.ErrUnexpectedEOF.
// Read more data and try again.
n, err := sd.dec.read()
if n == 0 {
// No more data, it is incomplete CBOR.
return 0, io.ErrUnexpectedEOF
}
if err != nil {
return 0, err
}
}
}
func (sd *StreamDecoder) updateState(bytesRead int) {
sd.remainingBytes -= bytesRead
if sd.remainingBytes < 0 {
sd.err = errors.New("remaining bytes are out of sync")
}
}
// NumBytesDecoded returns the accumulated number of bytes decoded using "DecodeXXX()"
func (sd *StreamDecoder) NumBytesDecoded() int {
return sd.decodedMsgBytes + sd.dec.d.off
}

2199
stream_decode_test.go Normal file

File diff suppressed because it is too large Load Diff

266
stream_encode.go Normal file
View File

@ -0,0 +1,266 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor
import (
"bytes"
"errors"
"io"
"math/big"
"reflect"
)
// ErrStreamClosed is the error indicating operations on a closed stream.
// E.g. any call to encode after a call to Close will return ErrStreamClosed.
var ErrStreamClosed = errors.New("cbor: operation on closed stream")
// StreamEncoder provides low-level API for sequential encoding.
//
// Users of StreamEncoder should be familiar with CBOR data models.
// When in doubt, please use Encoder instead.
//
// IMPORTANT: Use Valid() to check whether generated CBOR data is
// complete and well-formed.
type StreamEncoder struct {
*Encoder
buf *bytes.Buffer
closed bool
}
// NewStreamEncoder returns a new StreamEncoder for sequential encoding.
func NewStreamEncoder(w io.Writer) *StreamEncoder {
return defaultEncMode.NewStreamEncoder(w)
}
// Close closes StreamEncoder and subsequence operations (except for Close)
// will return ErrStreamClosed.
func (se *StreamEncoder) Close() {
if se.closed {
return
}
putEncodeBuffer(se.buf)
se.Encoder = nil
se.closed = true
}
// Flush writes streamed data to underlying io.Writer.
func (se *StreamEncoder) Flush() error {
if se.closed {
return ErrStreamClosed
}
_, err := se.buf.WriteTo(se.Encoder.w)
return err
}
// Encode writes the CBOR encoding of v.
func (se *StreamEncoder) Encode(v interface{}) error {
if err := se.checkIndefiniteLengthDataItemTypeIfNeeded(v); err != nil {
return err
}
return encode(se.buf, se.em, reflect.ValueOf(v))
}
// EncodeMapHead encodes CBOR map head of specified size.
func (se *StreamEncoder) EncodeMapHead(size uint64) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypeMap), size)
return nil
}
// EncodeArrayHead encodes CBOR array head of specified size.
func (se *StreamEncoder) EncodeArrayHead(size uint64) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypeArray), size)
return nil
}
// EncodeTagHead encodes CBOR tag head with num as tag number.
func (se *StreamEncoder) EncodeTagHead(num uint64) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypeTag), num)
return nil
}
// EncodeRawBytes writes b to the underlying writer.
// If b is an empty or nil byte slice, it is a no-op.
func (se *StreamEncoder) EncodeRawBytes(b []byte) error {
if se.closed {
return ErrStreamClosed
}
if len(b) == 0 {
return nil
}
se.buf.Write(b)
return nil
}
// EncodeNil encodes CBOR nil.
func (se *StreamEncoder) EncodeNil() error {
if se.closed {
return ErrStreamClosed
}
se.buf.Write(cborNil)
return nil
}
// EncodeBool encodes bool as CBOR bool.
func (se *StreamEncoder) EncodeBool(b bool) error {
if se.closed {
return ErrStreamClosed
}
bBytes := cborTrue
if !b {
bBytes = cborFalse
}
se.buf.Write(bBytes)
return nil
}
// EncodeUint encodes uint as CBOR positive integer.
func (se *StreamEncoder) EncodeUint(i uint) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypePositiveInt), uint64(i))
return nil
}
// EncodeUint8 encodes uint8 as CBOR positive integer.
func (se *StreamEncoder) EncodeUint8(i uint8) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypePositiveInt), uint64(i))
return nil
}
// EncodeUint16 encodes uint16 as CBOR positive integer.
func (se *StreamEncoder) EncodeUint16(i uint16) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypePositiveInt), uint64(i))
return nil
}
// EncodeUint32 encodes uint32 as CBOR positive integer.
func (se *StreamEncoder) EncodeUint32(i uint32) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypePositiveInt), uint64(i))
return nil
}
// EncodeUint64 encodes uint64 as CBOR positive integer.
func (se *StreamEncoder) EncodeUint64(i uint64) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypePositiveInt), i)
return nil
}
// EncodeInt encodes int as CBOR positive or negtive integer.
func (se *StreamEncoder) EncodeInt(i int) error {
return se.EncodeInt64(int64(i))
}
// EncodeInt8 encodes int8 as CBOR positive or negtive integer.
func (se *StreamEncoder) EncodeInt8(i int8) error {
return se.EncodeInt64(int64(i))
}
// EncodeInt16 encodes int16 as CBOR positive or negtive integer.
func (se *StreamEncoder) EncodeInt16(i int16) error {
return se.EncodeInt64(int64(i))
}
// EncodeInt32 encodes int32 as CBOR positive or negtive integer.
func (se *StreamEncoder) EncodeInt32(i int32) error {
return se.EncodeInt64(int64(i))
}
// EncodeInt64 encodes int64 as CBOR positive or negtive integer.
func (se *StreamEncoder) EncodeInt64(i int64) error {
if se.closed {
return ErrStreamClosed
}
t := cborTypePositiveInt
if i < 0 {
t = cborTypeNegativeInt
i = i*(-1) - 1
}
encodeHead(se.buf, byte(t), uint64(i))
return nil
}
// EncodeBytes encodes byte slice as CBOR byte string.
func (se *StreamEncoder) EncodeBytes(b []byte) error {
if se.closed {
return ErrStreamClosed
}
if b == nil {
se.buf.Write(cborNil)
return nil
}
encodeHead(se.buf, byte(cborTypeByteString), uint64(len(b)))
se.buf.Write(b)
return nil
}
// EncodeString encodes string as CBOR string.
func (se *StreamEncoder) EncodeString(s string) error {
if se.closed {
return ErrStreamClosed
}
encodeHead(se.buf, byte(cborTypeTextString), uint64(len(s)))
se.buf.WriteString(s)
return nil
}
var bigOne = big.NewInt(1)
// EncodeBigInt encodes big.Int as CBOR bignum (tag number 2 and 3).
func (se *StreamEncoder) EncodeBigInt(v *big.Int) error {
if se.closed {
return ErrStreamClosed
}
if se.em.bigIntConvert == BigIntConvertShortest {
if v.IsUint64() {
encodeHead(se.buf, byte(cborTypePositiveInt), v.Uint64())
return nil
}
if v.IsInt64() {
return se.EncodeInt64(v.Int64())
}
}
tagNum := 2
sign := v.Sign()
if sign < 0 {
tagNum = 3
// Create a new big.Int with value of -1 - v
v = new(big.Int).Abs(v)
v.Sub(v, bigOne)
}
b := v.Bytes()
// Write tag number
encodeHead(se.buf, byte(cborTypeTag), uint64(tagNum))
// Write bignum byte string
encodeHead(se.buf, byte(cborTypeByteString), uint64(len(b)))
se.buf.Write(b)
return nil
}

1164
stream_encode_test.go Normal file

File diff suppressed because it is too large Load Diff

View File

@ -37,7 +37,7 @@ func TestDecoder(t *testing.T) {
bytesRead := 0
for i := 0; i < 5; i++ {
for _, tc := range unmarshalTests {
var v interface{}
var v any
if err := decoder.Decode(&v); err != nil {
t.Fatalf("Decode() returned error %v", err)
}
@ -56,7 +56,7 @@ func TestDecoder(t *testing.T) {
}
for i := 0; i < 2; i++ {
// no more data
var v interface{}
var v any
err := decoder.Decode(&v)
if v != nil {
t.Errorf("Decode() = %v (%T), want nil (no more data)", v, v)
@ -106,7 +106,7 @@ func TestDecoderUnmarshalTypeError(t *testing.T) {
t.Errorf("NumBytesRead() = %v, want %v", decoder.NumBytesRead(), bytesRead)
}
var vi interface{}
var vi any
if err := decoder.Decode(&vi); err != nil {
t.Errorf("Decode() returned error %v", err)
}
@ -126,7 +126,7 @@ func TestDecoderUnmarshalTypeError(t *testing.T) {
}
for i := 0; i < 2; i++ {
// no more data
var v interface{}
var v any
err := decoder.Decode(&v)
if v != nil {
t.Errorf("Decode() = %v (%T), want nil (no more data)", v, v)
@ -162,7 +162,7 @@ func TestDecoderUnexpectedEOFError(t *testing.T) {
bytesRead := 0
for i := 0; i < len(unmarshalTests)-1; i++ {
tc := unmarshalTests[i]
var v interface{}
var v any
if err := decoder.Decode(&v); err != nil {
t.Fatalf("Decode() returned error %v", err)
}
@ -180,7 +180,7 @@ func TestDecoderUnexpectedEOFError(t *testing.T) {
}
for i := 0; i < 2; i++ {
// truncated data
var v interface{}
var v any
err := decoder.Decode(&v)
if v != nil {
t.Errorf("Decode() = %v (%T), want nil (no more data)", v, v)
@ -217,7 +217,7 @@ func TestDecoderReadError(t *testing.T) {
bytesRead := 0
for i := 0; i < len(unmarshalTests)-1; i++ {
tc := unmarshalTests[i]
var v interface{}
var v any
if err := decoder.Decode(&v); err != nil {
t.Fatalf("Decode() returned error %v", err)
}
@ -235,7 +235,7 @@ func TestDecoderReadError(t *testing.T) {
}
for i := 0; i < 2; i++ {
// truncated data because Reader returned error
var v interface{}
var v any
err := decoder.Decode(&v)
if v != nil {
t.Errorf("Decode() = %v (%T), want nil (no more data)", v, v)
@ -265,7 +265,7 @@ func TestDecoderNoData(t *testing.T) {
t.Run(tc.name, func(t *testing.T) {
decoder := NewDecoder(tc.reader)
for i := 0; i < 2; i++ {
var v interface{}
var v any
err := decoder.Decode(&v)
if v != nil {
t.Errorf("Decode() = %v (%T), want nil", v, v)
@ -280,12 +280,12 @@ func TestDecoderNoData(t *testing.T) {
func TestDecoderRecoverableReadError(t *testing.T) {
data := hexDecode("83010203") // [1,2,3]
wantValue := []interface{}{uint64(1), uint64(2), uint64(3)}
wantValue := []any{uint64(1), uint64(2), uint64(3)}
recoverableReaderErr := errors.New("recoverable reader error")
decoder := NewDecoder(newRecoverableReader(data, 1, recoverableReaderErr))
var v interface{}
var v any
err := decoder.Decode(&v)
if err != recoverableReaderErr {
t.Fatalf("Decode() returned error %v, want error %v", err, recoverableReaderErr)
@ -303,7 +303,7 @@ func TestDecoderRecoverableReadError(t *testing.T) {
}
// no more data
v = interface{}(nil)
v = any(nil)
err = decoder.Decode(&v)
if v != nil {
t.Errorf("Decode() = %v (%T), want nil (no more data)", v, v)
@ -317,13 +317,13 @@ func TestDecoderInvalidData(t *testing.T) {
data := []byte{0x01, 0x3e}
decoder := NewDecoder(bytes.NewReader(data))
var v1 interface{}
var v1 any
err := decoder.Decode(&v1)
if err != nil {
t.Errorf("Decode() returned error %v when decoding valid data item", err)
}
var v2 interface{}
var v2 any
err = decoder.Decode(&v2)
if err == nil {
t.Errorf("Decode() didn't return error when decoding invalid data item")
@ -645,7 +645,7 @@ func TestDecoderBuffered(t *testing.T) {
t.Errorf("Buffered() = 0x%x (%d bytes), want 0 bytes", buffered, len(buffered))
}
var v interface{}
var v any
err = decoder.Decode(&v)
if err != tc.decodeErr {
t.Errorf("Decode() returned error %v, want %v", err, tc.decodeErr)
@ -688,7 +688,7 @@ func TestEncoder(t *testing.T) {
func TestEncoderError(t *testing.T) {
testcases := []struct {
name string
value interface{}
value any
wantErrorMsg string
}{
{"channel cannot be marshaled", make(chan bool), "cbor: unsupported type: chan bool"},

View File

@ -18,9 +18,11 @@ type field struct {
typ reflect.Type
ef encodeFunc
ief isEmptyFunc
izf isZeroFunc
typInfo *typeInfo // used to decoder to reuse type info
tagged bool // used to choose dominant field (at the same level tagged fields dominate untagged fields)
omitEmpty bool // used to skip empty field
omitZero bool // used to skip zero field
keyAsInt bool // used to encode/decode field name as int
}
@ -157,7 +159,7 @@ func appendFields(
f := t.Field(i)
ft := f.Type
for ft.Kind() == reflect.Ptr {
for ft.Kind() == reflect.Pointer {
ft = ft.Elem()
}
@ -165,9 +167,11 @@ func appendFields(
continue
}
cborTag := true
tag := f.Tag.Get("cbor")
if tag == "" {
tag = f.Tag.Get("json")
cborTag = false
}
if tag == "-" {
continue
@ -177,7 +181,7 @@ func appendFields(
// Parse field tag options
var tagFieldName string
var omitempty, keyasint bool
var omitempty, omitzero, keyasint bool
for j := 0; tag != ""; j++ {
var token string
idx := strings.IndexByte(tag, ',')
@ -192,6 +196,10 @@ func appendFields(
switch token {
case "omitempty":
omitempty = true
case "omitzero":
if cborTag || jsonStdlibSupportsOmitzero {
omitzero = true
}
case "keyasint":
keyasint = true
}
@ -213,6 +221,7 @@ func appendFields(
idx: fIdx,
typ: f.Type,
omitEmpty: omitempty,
omitZero: omitzero,
keyAsInt: keyasint,
tagged: tagged})
} else {
@ -244,7 +253,7 @@ func getFieldValue(v reflect.Value, idx []int, f embeddedFieldNullPtrFunc) (fv r
fv = fv.Field(n)
if i < len(idx)-1 {
if fv.Kind() == reflect.Ptr && fv.Type().Elem().Kind() == reflect.Struct {
if fv.Kind() == reflect.Pointer && fv.Type().Elem().Kind() == reflect.Struct {
if fv.IsNil() {
// Null pointer to embedded struct field
fv, err = f(fv)

35
tag.go
View File

@ -1,3 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor
import (
@ -12,7 +15,7 @@ import (
// enclosed data item if it were to appear outside of a tag.
type Tag struct {
Number uint64
Content interface{}
Content any
}
// RawTag represents CBOR tag data, including tag number and raw tag content.
@ -23,11 +26,37 @@ type RawTag struct {
}
// UnmarshalCBOR sets *t with tag number and raw tag content copied from data.
//
// Deprecated: No longer used by this codec; kept for compatibility
// with user apps that directly call this function.
func (t *RawTag) UnmarshalCBOR(data []byte) error {
if t == nil {
return errors.New("cbor.RawTag: UnmarshalCBOR on nil pointer")
}
d := decoder{data: data, dm: defaultDecMode}
// Check if data is a well-formed CBOR data item.
// RawTag.UnmarshalCBOR() is exported, so
// the codec needs to support same behavior for:
// - Unmarshal(data, *RawTag)
// - RawTag.UnmarshalCBOR(data)
err := d.wellformed(false, false)
if err != nil {
return err
}
return t.unmarshalCBOR(data)
}
// unmarshalCBOR sets *t with tag number and raw tag content copied from data.
// This function assumes data is well-formed, and does not perform bounds checking.
// This function is called by Unmarshal().
func (t *RawTag) unmarshalCBOR(data []byte) error {
if t == nil {
return errors.New("cbor.RawTag: UnmarshalCBOR on nil pointer")
}
// Decoding CBOR null and undefined to cbor.RawTag is no-op.
if len(data) == 1 && (data[0] == 0xf6 || data[0] == 0xf7) {
return nil
@ -193,7 +222,7 @@ func (t *syncTagSet) Add(opts TagOptions, contentType reflect.Type, num uint64,
if contentType == nil {
return errors.New("cbor: cannot add nil content type to TagSet")
}
for contentType.Kind() == reflect.Ptr {
for contentType.Kind() == reflect.Pointer {
contentType = contentType.Elem()
}
tag, err := newTagItem(opts, contentType, num, nestedNum...)
@ -216,7 +245,7 @@ func (t *syncTagSet) Add(opts TagOptions, contentType reflect.Type, num uint64,
// Remove removes given tag content type from TagSet.
func (t *syncTagSet) Remove(contentType reflect.Type) {
for contentType.Kind() == reflect.Ptr {
for contentType.Kind() == reflect.Pointer {
contentType = contentType.Elem()
}
t.Lock()

View File

@ -1,3 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor
import (
@ -905,7 +908,7 @@ func TestDecodeWrongTag(t *testing.T) {
testCases := []struct {
name string
obj interface{}
obj any
data []byte
wantErrorMsg string
}{
@ -1240,7 +1243,7 @@ func TestMarshalRawTagWithEmptyContent(t *testing.T) {
}
func TestEncodeTag(t *testing.T) {
m := make(map[interface{}]bool)
m := make(map[any]bool)
m[10] = true
m[100] = true
m[-1] = true
@ -1295,7 +1298,7 @@ func TestDecodeTagToEmptyIface(t *testing.T) {
testCases := []struct {
name string
data []byte
wantObj interface{}
wantObj any
}{
{
name: "registered myBool",
@ -1326,7 +1329,7 @@ func TestDecodeTagToEmptyIface(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
var v1 interface{}
var v1 any
if err := dm.Unmarshal(tc.data, &v1); err != nil {
t.Errorf("Unmarshal() returned error %v", err)
}
@ -1334,7 +1337,7 @@ func TestDecodeTagToEmptyIface(t *testing.T) {
t.Errorf("Unmarshal to interface{} returned different values: %v, %v", tc.wantObj, v1)
}
var v2 interface{}
var v2 any
if err := dmSharedTags.Unmarshal(tc.data, &v2); err != nil {
t.Errorf("Unmarshal() returned error %v", err)
}
@ -1359,7 +1362,7 @@ func TestDecodeRegisteredTagToEmptyIfaceError(t *testing.T) {
data := hexDecode("d865d8663bffffffffffffffff") // 101(102(-18446744073709551616))
var v interface{}
var v any
if err := dm.Unmarshal(data, &v); err == nil {
t.Errorf("Unmarshal(0x%x) didn't return an error", data)
} else if _, ok := err.(*UnmarshalTypeError); !ok {
@ -1415,7 +1418,7 @@ func TestDecodeRegisterTagForUnmarshaler(t *testing.T) {
em, _ := EncOptions{}.EncModeWithTags(tags)
// Decode to empty interface. Unmarshal() should return object of registered type.
var v1 interface{}
var v1 any
if err := dm.Unmarshal(data, &v1); err != nil {
t.Errorf("Unmarshal() returned error %v", err)
}
@ -1448,7 +1451,7 @@ func TestDecodeRegisterTagForUnmarshaler(t *testing.T) {
func TestMarshalRawTagContainingMalformedCBORData(t *testing.T) {
testCases := []struct {
name string
value interface{}
value any
wantErrorMsg string
}{
// Nil RawMessage and empty RawMessage are encoded as CBOR nil.
@ -1536,3 +1539,125 @@ func TestEncodeBuiltinTag(t *testing.T) {
})
}
}
func TestUnmarshalRawTagOnBadData(t *testing.T) {
testCases := []struct {
name string
data []byte
errMsg string
}{
// Empty data
{
name: "nil data",
data: nil,
errMsg: io.EOF.Error(),
},
{
name: "empty data",
data: []byte{},
errMsg: io.EOF.Error(),
},
// Wrong CBOR types
{
name: "uint type",
data: hexDecode("01"),
errMsg: "cbor: cannot unmarshal positive integer into Go value of type cbor.RawTag",
},
{
name: "int type",
data: hexDecode("20"),
errMsg: "cbor: cannot unmarshal negative integer into Go value of type cbor.RawTag",
},
{
name: "byte string type",
data: hexDecode("40"),
errMsg: "cbor: cannot unmarshal byte string into Go value of type cbor.RawTag",
},
{
name: "string type",
data: hexDecode("60"),
errMsg: "cbor: cannot unmarshal UTF-8 text string into Go value of type cbor.RawTag",
},
{
name: "array type",
data: hexDecode("80"),
errMsg: "cbor: cannot unmarshal array into Go value of type cbor.RawTag",
},
{
name: "map type",
data: hexDecode("a0"),
errMsg: "cbor: cannot unmarshal map into Go value of type cbor.RawTag",
},
{
name: "primitive type",
data: hexDecode("f4"),
errMsg: "cbor: cannot unmarshal primitives into Go value of type cbor.RawTag",
},
{
name: "float type",
data: hexDecode("f90000"),
errMsg: "cbor: cannot unmarshal primitives into Go value of type cbor.RawTag",
},
// Truncated CBOR data
{
name: "truncated head",
data: hexDecode("18"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
// Truncated CBOR tag data
{
name: "truncated tag number",
data: hexDecode("d8"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
{
name: "tag number not followed by tag content",
data: hexDecode("da"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
{
name: "truncated tag content",
data: hexDecode("c074323031332d30332d32315432303a30343a3030"),
errMsg: io.ErrUnexpectedEOF.Error(),
},
// Extraneous CBOR data
{
name: "extraneous data",
data: hexDecode("c074323031332d30332d32315432303a30343a30305a00"),
errMsg: "cbor: 1 bytes of extraneous data starting at index 22",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Test RawTag.UnmarshalCBOR(data)
{
var v RawTag
err := v.UnmarshalCBOR(tc.data)
if err == nil {
t.Errorf("UnmarshalCBOR(%x) didn't return error", tc.data)
}
if !strings.HasPrefix(err.Error(), tc.errMsg) {
t.Errorf("UnmarshalCBOR(%x) returned error %q, want %q", tc.data, err.Error(), tc.errMsg)
}
}
// Test Unmarshal(data, *RawTag), which calls RawTag.unmarshalCBOR() under the hood
{
var v RawTag
err := Unmarshal(tc.data, &v)
if err == nil {
t.Errorf("Unmarshal(%x) didn't return error", tc.data)
}
if !strings.HasPrefix(err.Error(), tc.errMsg) {
t.Errorf("Unmarshal(%x) returned error %q, want %q", tc.data, err.Error(), tc.errMsg)
}
}
})
}
}