Motivation: half-precision floating point format is used to minimize storage
and traffic mostly. Application level manipulates with single and double
precision usually. So, two routines added to public API to encode/decode given
single precision value in the half precision format
Signed-off-by: S.Phirsov
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
See previous commit for more details. This just applies the same
technique using malloc/mmap to the the rest of the parser tests.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We can only validate_number() if we know that we have a number to
validate in the first place. If we've reached the end of our string, the
content that follows is not necessarily a number (it could be a Break
byte). More importantly, we could reach the end of the buffer.
This issue was masked by the way we provided data to the parser. It
always came from read-only memory becausee of QByteArray::fromRawData(),
so valgrind never caught any issues. Using QByteArray directly wouldn't
have helped because it always inserts a terminating null byte, which
always validates as a correct number (unsigned 0) and fails to trigger
valgrind.
So we need to use malloc() directly to make Valgrind complain. And there
was already a test that did:
==26543== Invalid read of size 1
==26543== at 0x483EA10: memmove (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==26543== by 0x43CEEA: read_bytes_unchecked (cborinternal_p.h:239)
==26543== by 0x43CFEC: extract_number_checked (cborinternal_p.h:286)
==26543== by 0x43D3E9: validate_number (cborvalidation.c:304)
==26543== by 0x43DC7B: validate_value (cborvalidation.c:551)
==26543== by 0x43DE8C: cbor_value_validate (cborvalidation.c:645)
==26543== by 0x4328D2: tst_Parser::strictValidation() (tst_parser.cpp:1637)
==26543== by 0x434632: tst_Parser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (tst_parser.moc:291)
==26543== by 0x4C0B36D: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.cpp:2310)
==26543== by 0x48673E9: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.h:122)
==26543== by 0x4860256: QTest::TestMethods::invokeTestOnData(int) const (qtestcase.cpp:922)
==26543== by 0x4860D4B: QTest::TestMethods::invokeTest(int, char const*, QTest::WatchDog*) const (qtestcase.cpp:1121)
==26543== Address 0x61c4db1 is 0 bytes after a block of size 1 alloc'd
==26543== at 0x483777F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==26543== by 0x403891: ParserWrapper::allocateMemory(unsigned long) (tst_parser.cpp:181)
==26543== by 0x436898: ParserWrapper::init(QByteArray const&) (tst_parser.cpp:126)
==26543== by 0x432712: tst_Parser::strictValidation() (tst_parser.cpp:1634)
==26543== by 0x434632: tst_Parser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (tst_parser.moc:291)
==26543== by 0x4C0B36D: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.cpp:2310)
==26543== by 0x48673E9: QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const (qmetaobject.h:122)
==26543== by 0x4860256: QTest::TestMethods::invokeTestOnData(int) const (qtestcase.cpp:922)
==26543== by 0x4860D4B: QTest::TestMethods::invokeTest(int, char const*, QTest::WatchDog*) const (qtestcase.cpp:1121)
==26543== by 0x4862083: QTest::TestMethods::invokeTests(QObject*) const (qtestcase.cpp:1465)
==26543== by 0x4862C14: QTest::qRun() (qtestcase.cpp:1903)
==26543== by 0x48626C3: QTest::qExec(QObject*, int, char**) (qtestcase.cpp:1792)
==26543==
PASS : tst_Parser::strictValidation(bytearray-0)
This commit goes further and makes it so an out-of-bounds access will
cause a pagefault.
Fixes#156.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The string chunk getter doesn't set ptr if there was an error
decoding. Instead, we need to deal with the error first.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
This updates to Qt 5.12, since Stephan has a build for that in his
PPA. But we need to turn off RDRNAD support for the Valgrind build, as
it doesn't understand that instruction.
This also downgrades GCC to 5.4, which is what comes by default with
Travis CI.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
fread(...) function returns the number of items read so far, not number of bytes, ergo, reading one st.st_size sized item produces 1 on success and 0 on failure.
Possibly fixing issue #77
Signed-off-by: phirsov <41143811+phirsov@users.noreply.github.com>
encode_half has been moved from compilersupport_p.h to cborinternal_p.h
in commit bfc40dcf90 so include this file
in json2cbor to avoid the following build failure:
/home/buildroot/autobuild/run/instance-0/output/host/bin/microblazeel-linux-gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I./src -std=gnu99 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -c -o tools/json2cbor/json2cbor.o tools/json2cbor/json2cbor.c
tools/json2cbor/json2cbor.c: In function 'decode_json_with_metadata':
tools/json2cbor/json2cbor.c:295:50: warning: implicit declaration of function 'encode_half' [-Wimplicit-function-declaration]
(half = encode_half(v), cbor_encode_half_float(encoder, &half));
^~~~~~~~~~~
/home/buildroot/autobuild/run/instance-0/output/host/bin/microblazeel-linux-gcc -o bin/json2cbor tools/json2cbor/json2cbor.o lib/libtinycbor.so -lcjson -lm
tools/json2cbor/json2cbor.o: In function `decode_json_with_metadata':
(.text+0xe54): undefined reference to `encode_half'
collect2: error: ld returned 1 exit status
Makefile:151: recipe for target 'bin/json2cbor' failed
Fixes:
- http://autobuild.buildroot.net/results/afd8d24f2a4e501264abff618cf421d4bd088ebf
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
I ported this from code I had written for Qt, where the subtraction was
correct. Qt's code is roughly:
uchar b = *src++;
...
int bytesAvailable = Traits::availableBytes(src, end);
if (Q_UNLIKELY(bytesAvailable < charsNeeded - 1)) {
Which means src was advanced by the time we call Traits::availableBytes,
meaning that call returns the number of continuation bytes only. Our
code was:
ptrdiff_t n = end - *buffer;
...
uc = *(*buffer)++;
...
if (n < charsNeeded - 1)
Which means our n is the total number of bytes, including the first
byte.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
... and put it behind #ifndef CBOR_NO_FLOATING_POINT.
The functions were not really compiler support, but are internal
functionality of TinyCBOR. This commit also puts both the <math.H> and
<float.h> headers behind an #if, so constrained systems without floating
point support won't enable it.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
The hosted version makes use of math functions (usually, we don't test
that CBOR_NO_FLOATING_POINT is defined).
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
On the Mac, the Clang compiler complains about our (required) use of
uint64_t, which is typedef'ed to long long. That type didn't exist in
C90.
On Windows, I forgot the CONFIG += console, resulting in
error LNK2019: unresolved external symbol _WinMain@16 referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ)
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Because of all the inline functions, #include'ing <cbor.h> without
linking in all .c files may result in undefined symbol linker errors.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
This fixes the issues in all the headers (even in the private headers),
but not all of them in the .c files. I only fixed those that didn't make
the code look much uglier or those for which GCC complains with
-std=c90. The compiler only warns about mixed mixed variable
declarations and code, unless you use -pedantic-errors.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Turns out that we were relying on undefined behaviour when doing
(uint64_t)fabs(val) == fabs(val)
This problem was noted when running our test data on an AArch64 build.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Adds ifdefs around the float handling to return an CborErrorUnknownType
if compiled without floating point support
Signed-off-by: Koen Zandberg <koen@bergzand.net>
Compilation for 8 bit architectures fails due to a bit shift larger than
8 for a uint8_t data type. This commit changes the data type to uint32_t
for the tag array.
Signed-off-by: Koen Zandberg <koen@bergzand.net>
So we can tell from the output if the library is increasing or not.
This also fixes a silly bug in testing non-tags, introduced by
0170ebdcdc.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Following Edward Welbourne's recommendation, first we remove the comments:
$ doxygen -s -u
Then we compare to the stock Doxyfile without comments (doxygen -s -g)
and save only what's to our file:
$ doxygen -s -g - | comm -13 Doxyfile > Doxyfile.new
$ mv Doxyfile.new Doxyfile
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
We don't need to compare the map lengths directly. The memcmp is
sufficient, since the source data is big endian.
Of course, verifying for sorting requires the map has known length.
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
phirsov