The build fails after the transition to Apple silicon and I don't know
why:
```
: && /usr/bin/clang -Oz -g -Werror -arch arm64 -Wl,-search_paths_first -Wl,-headerpad_max_install_names tools/json2cbor/CMakeFiles/json2cbor.dir/json2cbor.c.o -o tools/json2cbor/json2cbor libtinycbor.a -lcjson -lm && :
ld: library 'cjson' not found
```
Homebrew says it installed and CMake says it found it.
```
-- Checking for module 'libcjson'
-- Found libcjson, version 1.7.19
```
I won't investigate.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Instead of saying Linux (a.k.a. glibc) has it and Apple has funopen(),
use the fact that we've just detected them and inform the .c source
which one it was.
Fixes#306
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Instead of performing a lossy conversion to double and printing that.
It's irrelevant whether the parser on the other side can store this
precision, only that it can parse this. That includes numbers outside
the range of int64_t, which CBOR does support.
We do this by simply removing code from cbortojson.c and instead just
relying on what cborpretty.c already has.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We weren't dereferencing the variable, so this should have been safe.
However, it isn't clear in the C and C++ standards whether it
legitimately was safe. So let's just fix it.
```
simplereader.c:180:9: warning: pointer ‘buf’ may be used after ‘free’ [-Wuse-after-free]
simplereader.c:177:5: note: call to ‘free’ here
```
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
I think GCC says `__has_cpp_attribute(fallthrough)` is true because C++
supports it (it means "has C++ attribute"), but that doesn't apply to
the C language. This causes a compilation error:
```
compilersupport_p.h:57:41: error: expected expression before '[' token
57 | # define CBOR_FALLTHROUGH [[fallthrough]]
| ^
cborparser.c:225:13: note: in expansion of macro 'CBOR_FALLTHROUGH'
225 | CBOR_FALLTHROUGH;
| ^~~~~~~~~~~~~~~~
```
Instead, we should use the C23 `__has_c_attribute` to detect the C
attribute.
Fixes#293.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The static builds with -Os or -Oz are there only so we get the library
size in the output. For testing, let's compile as a shared library and
properly in debug mode.
For Linux, we've had Valgrind. For macOS, this is now an ASan build.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
TinyCBOR has been at version 0.x for 10 years. This commit declares we
are now quite mature. I'm skipping version 1.0 through 6.0 and simply
calling the 0.7.0 to be 7.0.
Since this is binary compatible with the 0.6 version, I'm keeping the
SONAME of the shared library simply "libtinycbor.so.0" (note: previous
SONAME was "libtinycbor.so.0.6").
The Makefile build is not updated.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
As the comment says, to ensure we don't accidentally use something from
a new edition of the language than C99. In 2015 I think compilers still
defaulted to C99, but now in 2025, they default to C17.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Both the C and C++ standards say it is Undefined Behavior to convert a
floating point number to integer if the input is out of bounds of the
destination type.
And indeed this started failing in recent builds, with
val = 18446744073709551616 (2^64)
it has probably been producing ival = 18446744073709551615 for a while,
but the conversion back to floating point now rounded up and compared
equal to the input.
So let's just fix it.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
And make CBOR_PRIVATE_API fall back to it.
We also need to provide the file for the old Makefile build too.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We were returning from the function with the memory we had allocated and
freed, if the second iteration over the string produced a failure that
didn't happen on the first one. This can't happen with pure memory
buffers, but can happen with an external data source that fails to
produce the same contents twice.
I'm documenting that the values in all error conditions except for OOM
are undefined, so one mustn't attempt to use them, even to free. This
does not change behaviour of the library, just documents.
But this commit does make it clear the OOM condition will return a valid
`*buflen` and `next`, the latter of which is new behaviour with this
commit.
Fixes#258.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
There's a discussion in the C and C++ communities whether you're allowed
to use the values of pointers that have been deallocated, if you don't
dereference them. Some argue that it is Undefined Behaviour in spite of
the numeric value stored in the variable not having changed.
Instead of arguing, let's just make sure we don't use the pointers after
they have become dangling. We only needed the offset of how far we've
written into the buffer to restore the state and we have a function that
returns exactly that.
Seen while debugging #259.
Drive-by keep the `buffersize` global variable unchanged until after
`realloc()` has returned with success.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We hadn't bothered, as this was just example-like code to show how one
could convert from CBOR to JSON. But as it was added to the library (no
extra dependency), we should Do The Right Thing (DTRT) and escape.
This patch could have used cbor_value_to_pretty() to print the string,
which has better support for UTF-8 escaping and thus checks for UTF-8
correctness, but that would make map_to_json()'s metadata functionality
much more complex, especially since we cannot rely on open_memstream()
always being available. Therefore, we are partially duplicating
cborpretty.c's utf8EscapedDump().
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
1024 levels will probably be good enough for everyone, like
cborparser.c. For those for whom it isn't, they can set the limit during
the build.
We already had this for the plain parser, so TinyCBOR wouldn't cause a
stack overflow in case of a malformed stream (intentionally or not) when
simply parsing and advancing over the stream. This same protection
wasn't applied to the content converting from CBOR to JSON.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
`Q_ASSERT()` disappears in release mode, leading Clang to print a static
analysis warning about an impossible condition:
```
tst_parser.cpp:251:16: warning: variable 'err' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
251 | } else if (ourType == CborTextStringType) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tst_parser.cpp:263:12: note: uninitialized use occurs here
263 | return err;
| ^~~
tst_parser.cpp:251:12: note: remove the 'if' if its condition is always true
```
`Q_UNREACHABLE()` becomes a plain `__builtin_unreachable()` in release
mode.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The /* fallthrough */ comment isn't always handled, causing some
compilers to complain about falling through.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Homebrew no longer carries precompiled versions of Qt for macos-11, so
switch to macos-13, the last on x86 CPUs (so we can still run
Valgrind). It's also going away after the end of June.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We don't support this any more.
```
../../src/cbor.h:255:69: error: '_Bool' is a C99 extension [-Werror,-Wc99-extensions]
CBOR_INLINE_API CborError cbor_encode_boolean(CborEncoder *encoder, bool value)
^
```
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
It doesn't know -Wdiscarded-qualifiers. Reported by @pjonsson on #247.
I really need to switch to CMake - #242
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We had tests for cbor_validate* API for the errors it would find, but not
that it did correctly reported CborNoError where no errors were
expected.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Instead of just one function (_cbor_value_get_string_chunk), we now have
_cbor_value_begin_string_iteration, _cbor_value_finish_string_iteration,
_cbor_value_get_string_chunk_size, and _cbor_value_get_string_chunk.
The "begin" function positions the pointer at the first chunk. That's
what makes "get_size" possible, since it doesn't need to check for any
state. The "finish" funcntion allows the caller to distinguish an error
parsing the string from an error parsing the next value.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We need to re-parse if the input buffer was too short to read the
current element's information. When that happens, the current element
will be CborInvalidType, so we can't easily resume.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Instead of consuming it at the end of the last element of a map or array
of unknown length. This allows us to obtain the pointer to or offset of
the Break byte.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The extract_length() was the only case that called
extract_number_and_advance() without verifying we had a proper number. This
commit reworks the implementation so extract_number_and_advance() reuses
the number previously read by preparse_value(), and extract_length() gets
inlined to the only place that uses it.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
It was originally non-inline because I had thought of doing conversions
from half-float to float and onwards to double, but I never actually
made that in the API. Instead, even the get_float() and get_double(), we
only memcpy anyway and leave it up to the upper layer to convert, as
needed.
This change triggered an use-when-uninitialised false positive warning
that I needed to work around in the validator.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We don't need to skimp on bits in the CborValue::flags, so save the fact
that the preparser found a 64-bit number in there. This saves us from
having to re-read the descriptor byte again in
_cbor_value_decode_int64_internal().
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
This commit does not change all the readers yet, this is just the first
step.
As an interesting side-effect, we ended up reading the half-float into
it->extra and need not re-read it. The cbor_value_get_half_float()
function can be inlined in a later commit.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
As noted in the comment, we need to be sure we don't allow a length too
big from the stream to overflow and become smaller than the number of
bytes we're looking for.
This is also the first step in creating an API that reads from something
other than a linear buffer.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
QCOMPARE macro has a return in case of failure. If a test fails inside
the encodeOne function, we log that error, but were proceeding to
perform more tests (which could fail again and produce more errors).
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Because of all the inline functions, #include'ing <cbor.h> without
linking in all .c files may result in undefined symbol linker errors.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We don't need to compare the map lengths directly. The memcmp is
sufficient, since the source data is big endian.
Of course, verifying for sorting requires the map has known length.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The last version of Qt to support MSVC 2015 is no longer maintained, so
I'm skipping that. I'm therefore rededicating MSVC 2017 for 32-bit.
There's also no more no-tests build.
MSVC (and I think ICC too) are lacking the simpler, scalar instructions
to convert from single-precision to half-precision and back. Instead, we
need to use the packed data intrinsics.
Fixes#192.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
If a map end (Break byte) occurs before we've read the concrete item for
the value, then the map is invalid.
Fixes#167
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
QCOMPARE macro has a return in case of failure. If a test fails inside
the encodeOne function, we log that error, but were proceeding to
perform more tests (which could fail again and produce more errors).
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
See previous commit for more details. This just applies the same
technique using malloc/mmap to the the rest of the parser tests.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We can only validate_number() if we know that we have a number to
validate in the first place. If we've reached the end of our string, the
content that follows is not necessarily a number (it could be a Break
byte). More importantly, we could reach the end of the buffer.
This issue was masked by the way we provided data to the parser. It
always came from read-only memory becausee of QByteArray::fromRawData(),
so valgrind never caught any issues. Using QByteArray directly wouldn't
have helped because it always inserts a terminating null byte, which
always validates as a correct number (unsigned 0) and fails to trigger
valgrind.
So we need to use malloc() directly to make Valgrind complain. And there
was already a test that did:
==26543== Invalid read of size 1
==26543== at 0x483EA10: memmove (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==26543== by 0x43CEEA: read_bytes_unchecked (cborinternal_p.h:239)
==26543== by 0x43CFEC: extract_number_checked (cborinternal_p.h:286)
==26543== by 0x43D3E9: validate_number (cborvalidation.c:304)
==26543== by 0x43DC7B: validate_value (cborvalidation.c:551)
==26543== by 0x43DE8C: cbor_value_validate (cborvalidation.c:645)
==26543== by 0x4328D2: tst_Parser::strictValidation() (tst_parser.cpp:1637)
==26543== by 0x434632: tst_Parser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (tst_parser.moc:291)
==26543== by 0x4C0B36D: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.cpp:2310)
==26543== by 0x48673E9: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.h:122)
==26543== by 0x4860256: QTest::TestMethods::invokeTestOnData(int) const (qtestcase.cpp:922)
==26543== by 0x4860D4B: QTest::TestMethods::invokeTest(int, char const*, QTest::WatchDog*) const (qtestcase.cpp:1121)
==26543== Address 0x61c4db1 is 0 bytes after a block of size 1 alloc'd
==26543== at 0x483777F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==26543== by 0x403891: ParserWrapper::allocateMemory(unsigned long) (tst_parser.cpp:181)
==26543== by 0x436898: ParserWrapper::init(QByteArray const&) (tst_parser.cpp:126)
==26543== by 0x432712: tst_Parser::strictValidation() (tst_parser.cpp:1634)
==26543== by 0x434632: tst_Parser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (tst_parser.moc:291)
==26543== by 0x4C0B36D: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.cpp:2310)
==26543== by 0x48673E9: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.h:122)
==26543== by 0x4860256: QTest::TestMethods::invokeTestOnData(int) const (qtestcase.cpp:922)
==26543== by 0x4860D4B: QTest::TestMethods::invokeTest(int, char const*, QTest::WatchDog*) const (qtestcase.cpp:1121)
==26543== by 0x4862083: QTest::TestMethods::invokeTests(QObject*) const (qtestcase.cpp:1465)
==26543== by 0x4862C14: QTest::qRun() (qtestcase.cpp:1903)
==26543== by 0x48626C3: QTest::qExec(QObject*, int, char**) (qtestcase.cpp:1792)
==26543==
PASS : tst_Parser::strictValidation(bytearray-0)
This commit goes further and makes it so an out-of-bounds access will
cause a pagefault.
Fixes#156.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The string chunk getter doesn't set ptr if there was an error
decoding. Instead, we need to deal with the error first.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
This updates to Qt 5.12, since Stephan has a build for that in his
PPA. But we need to turn off RDRNAD support for the Valgrind build, as
it doesn't understand that instruction.
This also downgrades GCC to 5.4, which is what comes by default with
Travis CI.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
I ported this from code I had written for Qt, where the subtraction was
correct. Qt's code is roughly:
uchar b = *src++;
...
int bytesAvailable = Traits::availableBytes(src, end);
if (Q_UNLIKELY(bytesAvailable < charsNeeded - 1)) {
Which means src was advanced by the time we call Traits::availableBytes,
meaning that call returns the number of continuation bytes only. Our
code was:
ptrdiff_t n = end - *buffer;
...
uc = *(*buffer)++;
...
if (n < charsNeeded - 1)
Which means our n is the total number of bytes, including the first
byte.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Thiago Macieira