Motivation: half-precision floating point format is used to minimize storage
and traffic mostly. Application level manipulates with single and double
precision usually. So, two routines added to public API to encode/decode given
single precision value in the half precision format
Signed-off-by: S.Phirsov
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
MSVC (and I think ICC too) are lacking the simpler, scalar instructions
to convert from single-precision to half-precision and back. Instead, we
need to use the packed data intrinsics.
Fixes#192.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
When a container is closed, the outer container's buffer state is
synchronised with the inner container's state.
This was previously done via:
```
CborError cbor_encoder_close_container(CborEncoder *encoder, const CborEncoder *containerEncoder)
{
if (encoder->end)
encoder->data.ptr = containerEncoder->data.ptr;
else
encoder->data.bytes_needed = containerEncoder->data.bytes_needed;
encoder->end = containerEncoder->end;
...
return CborNoError;
}
```
However, strictly speaking the inner container could have updated `end` to be `NULL` if
the buffer was too small. In that case, the outer container should carry over `bytes_needed`
and not `ptr`. However, the logic was using the outer container's "stale" view of `end`.
However, generally `sizeof(ptr)` and `sizeof(bytes_needed)` are the same, and these values
exist in a union. Therefore, the correct values where still being copied.
This change moves the synchronisation of `end` to ahead of the synchronisation of `data`.
It additionally, actually avoids this potential error by simply synchronising the full `data`
structure, thereby avoiding the conditional logic. Simplifying the code to simply:
```
encoder->end = containerEncoder->end;
encoder->data = containerEncoder->data;
```
I was getting the following error when compiling cborencoder.c as part of
QtBase for FreeBSD MIPS64:
error: explicitly assigning value of variable of type 'uint64_t' (aka 'unsigned long') to itself [-Werror,-Wself-assign]
v = cbor_htonll(v);
Signed-off-by: Alex Richardson <Alexander.Richardson@cl.cam.ac.uk>
The open_memstream.c was using GLIBC macro definition to test if the
library is building on a Linux box. This makes impossible to build
tinycbor against other C libraries, as musl for example.
Signed-off-by: Ricardo Crudo <ricardo.crudo@gmail.com>
bytestring got escaped hex string literal output format to not to confuse e.g. "\x11" bytestring with 11 integer
Signed-off-by: phirsov <41143811+phirsov@users.noreply.github.com>
cbortojson.c: initialization discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
Signed-off-by: Mathieu Stephan <contributors@themooltipass.com>
If a map end (Break byte) occurs before we've read the concrete item for
the value, then the map is invalid.
Fixes#167
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Fixed the error checking example proposed in the comments/documentation at the beginning of the file.
Changed occurrences of `if(!err)` to `if(err)`, and `cbor_assert(err)` to `cbor_assert(!err)`.
Signed-off-by: Elie El Khoury <eliekhoury@hotmail.de>
See previous commit for more details. This just applies the same
technique using malloc/mmap to the the rest of the parser tests.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We can only validate_number() if we know that we have a number to
validate in the first place. If we've reached the end of our string, the
content that follows is not necessarily a number (it could be a Break
byte). More importantly, we could reach the end of the buffer.
This issue was masked by the way we provided data to the parser. It
always came from read-only memory becausee of QByteArray::fromRawData(),
so valgrind never caught any issues. Using QByteArray directly wouldn't
have helped because it always inserts a terminating null byte, which
always validates as a correct number (unsigned 0) and fails to trigger
valgrind.
So we need to use malloc() directly to make Valgrind complain. And there
was already a test that did:
==26543== Invalid read of size 1
==26543== at 0x483EA10: memmove (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==26543== by 0x43CEEA: read_bytes_unchecked (cborinternal_p.h:239)
==26543== by 0x43CFEC: extract_number_checked (cborinternal_p.h:286)
==26543== by 0x43D3E9: validate_number (cborvalidation.c:304)
==26543== by 0x43DC7B: validate_value (cborvalidation.c:551)
==26543== by 0x43DE8C: cbor_value_validate (cborvalidation.c:645)
==26543== by 0x4328D2: tst_Parser::strictValidation() (tst_parser.cpp:1637)
==26543== by 0x434632: tst_Parser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (tst_parser.moc:291)
==26543== by 0x4C0B36D: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.cpp:2310)
==26543== by 0x48673E9: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.h:122)
==26543== by 0x4860256: QTest::TestMethods::invokeTestOnData(int) const (qtestcase.cpp:922)
==26543== by 0x4860D4B: QTest::TestMethods::invokeTest(int, char const*, QTest::WatchDog*) const (qtestcase.cpp:1121)
==26543== Address 0x61c4db1 is 0 bytes after a block of size 1 alloc'd
==26543== at 0x483777F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==26543== by 0x403891: ParserWrapper::allocateMemory(unsigned long) (tst_parser.cpp:181)
==26543== by 0x436898: ParserWrapper::init(QByteArray const&) (tst_parser.cpp:126)
==26543== by 0x432712: tst_Parser::strictValidation() (tst_parser.cpp:1634)
==26543== by 0x434632: tst_Parser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (tst_parser.moc:291)
==26543== by 0x4C0B36D: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.cpp:2310)
==26543== by 0x48673E9: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.h:122)
==26543== by 0x4860256: QTest::TestMethods::invokeTestOnData(int) const (qtestcase.cpp:922)
==26543== by 0x4860D4B: QTest::TestMethods::invokeTest(int, char const*, QTest::WatchDog*) const (qtestcase.cpp:1121)
==26543== by 0x4862083: QTest::TestMethods::invokeTests(QObject*) const (qtestcase.cpp:1465)
==26543== by 0x4862C14: QTest::qRun() (qtestcase.cpp:1903)
==26543== by 0x48626C3: QTest::qExec(QObject*, int, char**) (qtestcase.cpp:1792)
==26543==
PASS : tst_Parser::strictValidation(bytearray-0)
This commit goes further and makes it so an out-of-bounds access will
cause a pagefault.
Fixes#156.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The string chunk getter doesn't set ptr if there was an error
decoding. Instead, we need to deal with the error first.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
This updates to Qt 5.12, since Stephan has a build for that in his
PPA. But we need to turn off RDRNAD support for the Valgrind build, as
it doesn't understand that instruction.
This also downgrades GCC to 5.4, which is what comes by default with
Travis CI.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
fread(...) function returns the number of items read so far, not number of bytes, ergo, reading one st.st_size sized item produces 1 on success and 0 on failure.
Possibly fixing issue #77
Signed-off-by: phirsov <41143811+phirsov@users.noreply.github.com>
encode_half has been moved from compilersupport_p.h to cborinternal_p.h
in commit bfc40dcf90 so include this file
in json2cbor to avoid the following build failure:
/home/buildroot/autobuild/run/instance-0/output/host/bin/microblazeel-linux-gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I./src -std=gnu99 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -c -o tools/json2cbor/json2cbor.o tools/json2cbor/json2cbor.c
tools/json2cbor/json2cbor.c: In function 'decode_json_with_metadata':
tools/json2cbor/json2cbor.c:295:50: warning: implicit declaration of function 'encode_half' [-Wimplicit-function-declaration]
(half = encode_half(v), cbor_encode_half_float(encoder, &half));
^~~~~~~~~~~
/home/buildroot/autobuild/run/instance-0/output/host/bin/microblazeel-linux-gcc -o bin/json2cbor tools/json2cbor/json2cbor.o lib/libtinycbor.so -lcjson -lm
tools/json2cbor/json2cbor.o: In function `decode_json_with_metadata':
(.text+0xe54): undefined reference to `encode_half'
collect2: error: ld returned 1 exit status
Makefile:151: recipe for target 'bin/json2cbor' failed
Fixes:
- http://autobuild.buildroot.net/results/afd8d24f2a4e501264abff618cf421d4bd088ebf
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
I ported this from code I had written for Qt, where the subtraction was
correct. Qt's code is roughly:
uchar b = *src++;
...
int bytesAvailable = Traits::availableBytes(src, end);
if (Q_UNLIKELY(bytesAvailable < charsNeeded - 1)) {
Which means src was advanced by the time we call Traits::availableBytes,
meaning that call returns the number of continuation bytes only. Our
code was:
ptrdiff_t n = end - *buffer;
...
uc = *(*buffer)++;
...
if (n < charsNeeded - 1)
Which means our n is the total number of bytes, including the first
byte.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
... and put it behind #ifndef CBOR_NO_FLOATING_POINT.
The functions were not really compiler support, but are internal
functionality of TinyCBOR. This commit also puts both the <math.H> and
<float.h> headers behind an #if, so constrained systems without floating
point support won't enable it.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The hosted version makes use of math functions (usually, we don't test
that CBOR_NO_FLOATING_POINT is defined).
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Thiago Macieira