Commit Graph
76 Commits
Author SHA1 Message Date
Thiago Macieira 57bcf4fec4 Add support for IEEE 754 half-precision floating point
The CBOR spec has support for them, but we didn't take them into account
in the CBOR and JSON dumpers nor the JSON parser.

See: https://en.wikipedia.org/wiki/Half-precision_floating-point_format

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 18:50:27 -07:00
Thiago Macieira 78632b3e5c json2cbor: Implement parsing of metadata
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 18:50:27 -07:00
Thiago Macieira d4c9ecb91f json2cbor: Add the tool to convert from JSON to CBOR
Conversion works without metadata interpretation yet. That's coming
next.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 18:50:27 -07:00
Thiago Macieira 2a5fb79dae Fix build on OS X
OS X has no open_memstream and its linker has no --gc-sections.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 14:15:48 -07:00
Thiago Macieira 8c00769296 Add support for dumping CBOR to JSON 2015-09-29 14:15:48 -07:00
Thiago Macieira 7a0ddb83e2 CBOR-to-JSON: Add support for saving CBOR metadata in JSON
Metadata is saved only when the item in question is part of a map,
otherwise we don't have anywhere to save the metadata.

The metadata consists of an extra JSON object, whose key is the same key
with "$cbor" appended. The possible entries in that map are:
 - tag: the innermost tag before the actual type
 - t:   the original CBOR type (CborType)
 - v:   supplemental value to reconstruct the original value
        for simple types, it's the simple type (0-255)
        for integers, it's the sign followed by the hex dump
        for floating point, it's "nan", "inf" or "-inf"

In addition, if CborConvertStringifyMapKeys is active and the key itself
wasn't a string, an extra object entry with the same string key followed
by "$keycbordump" is added, indicating that the key was stringified.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 14:15:48 -07:00
Thiago Macieira b4b44b57b9 CBOR-to-JSON: convert the stringification of map keys to CBOR dumps
This avoids having a third set of CborValue-to-text conversion
functions. What's more, the CBOR dump format is unique and reversible
(except for inf & nan, which lose their precision indication).

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 14:14:34 -07:00
Thiago Macieira 782f2545a0 CBOR-to-JSON: add support for CBOR tags
Normally, only three tags are handled and only for byte strings. But if
you set the CborConvertTagsToObjects flag, then all tags are transformed
into JSON objects (and the byte string-specific tags don't get
honoured).

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 14:14:34 -07:00
Thiago Macieira 2b1800c399 Add initial CBOR-to-JSON conversion code
Tags are not handled yet.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-29 14:14:24 -07:00
Thiago Macieira 4e9626cd16 Implement checking of array and map sizes
This fixes issue #4. The code is placed in a new file and function so
that only people who need it will pay the cost of this function. The
impact to existing code is minimal.

Before:
   text    data     bss     dec     hex filename
    945       0       0     945     3b1 cborencoder.o
   2743       0       0    2743     ab7 cborparser.o
After:
   1026       0       0    1026     424 cborencoder.o
   2756       0       0    2756     ac4 cborparser.o

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-21 16:47:21 -07:00
Thiago Macieira 510e5b807c encode_number: Ensure 64-bit alignment
This causes the 64-bit store to be aligned. On x86 and x86-64 this
should improve performance; on some other architectures, this should
prevent a byte-by-byte copy due to the misalignment.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-21 16:07:57 -07:00
Thiago Macieira 397f97979a Don't tempt undefined behaviour in incrementing null pointers
When we wrote:
       encoder->end = encoder->ptr = NULL;
    }
    ++encoder->ptr;

We're incrementing a null pointer. This could be considered undefined
behaviour and the compiler would be free to remove the increment.

This commit solves that by using an integer called bytes_needed to store
the number of bytes needed.

The trick is, however, that this commit is actually a no-op: the new
member occupies the same storage as ptr and in all of the new
conditionals, both branches evaluate to the same code, so the
conditional can be removed.

The code is complex like this to ensure we don't run afoul of another
undefined behaviour: reading from a member of an union that was not the
last written to.

CID: 306167
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-20 17:45:14 -07:00
Thiago Macieira 5663524401 Micro-optimisation: reorder the parameters to create_container
This makes the create_array and map functions simpler. The noinline
forces the compiler not to inline create_container. There's a net gain
this way.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-20 17:45:14 -07:00
Thiago Macieira f0ce0647f3 Fix/simplify printing of floating point numbers that are integer
This avoids writing to a buffer and scanning it. As an advantage, we
always print full precision when it matches an integer.

Incidentally, let's make sure we use the C11 macro that has the correct
number of decimal digits a double-precision floating-point can hold.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-18 16:46:58 -07:00
Thiago Macieira 9098b8b7d0 Fix Makefile: cbortojson.c doesn't exist (yet)
It's coming.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-18 16:44:30 -07:00
Thiago Macieira 7dc8cebde9 Micro-optimise cbor_value_get_{float,double}
floats and double are too big for value->extra and the
CborIteratorFlag_IntegerValueTooLarge flag is always set (see
cborparser.c:174). So there's no point in checking for the flag and we
can instead go straight to the internal function.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-17 22:35:27 -07:00
Thiago Macieira 4b916b484e Fix "line 0: test: too many arguments" error
I forgot the quoting...

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-17 22:35:27 -07:00
Thiago Macieira b2a7f3ee3c Add .gitattributes to make the distribution files cleaner
Don't include .gitignore and .gitattributes itself, but do include the
SHA-1 of the commit in the .tag file.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-16 12:11:20 -07:00
Thiago Macieira 07716ef9a0 Add a "make release" target
Release steps:
 1) run distcheck
 2) update the version number && commit it
 3) create a new tag
 4) create the release files

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-16 12:11:20 -07:00
Thiago Macieira cd724957e1 Use libtinycbor.a from the unit tests too
We then avoid recompiling every time. Exceptions:
 - tst_parser requires a recursion limit set
 - tst_cpp #includes so that the sources are compiled as C++

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-16 12:11:20 -07:00
Thiago Macieira 12b8960dde Add a buildsystem to tinycbor
The top-level Makefile will build the library, the cbordump tool and possibly
install them. Along with the buildsystem, tinycbor also gets a pkgconfig file
and a version number.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-16 12:11:19 -07:00
Thiago Macieira 92d8c69e70 Fix build with ICC: __builtin_add_overflow isn't defined
compilersupport_p.h(137): warning #266: function "__builtin_add_overflow" declared implicitly

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-16 07:50:41 -07:00
Leandro Pereira 3d866c212f Remove shadow from type variable in value_to_pretty()
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-09-16 07:50:18 -07:00
Thiago Macieira 932e08559b Fix conversion of large negative integers to string
There was an off-by-one error due to the way we handled them.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Erich Keane <erich.keane@intel.com>
2015-09-15 15:09:19 -07:00
Thiago Macieira 1599f0578d Merge pull request #7 from lpereira/stringz
Add convenience function cbor_encode_text_stringz()
2015-09-15 14:53:32 -07:00
Leandro Pereira 12415612e1 Add convenience function cbor_encode_text_stringz()
This calculates the length of a text string to be encoded using
strlen().  Useful when encoding constant strings.

Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
2015-09-15 17:28:45 -03:00
Erich Keane 47a78569c0 Allow TinyCbor to continue during a OOM situation
This patch causes TinyCbor Encoder to remain in a valid-state in the situation
where it has overrun its buffer.  It does this by setting the pointer
and end variable of the encoder to NULL, making encoder->ptr - NULL (or
encoder->ptr - encoder->end) equal the amount of additional data
necessary to properly encode the package.

Signed-off-by: Erich Keane <erich.keane@intel.com>
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
v0.1
2015-08-11 13:34:04 -07:00
Thiago Macieira 3e83c0dc6e Add a tool that prints a human-readable decoding of a CBOR stream
It prints exactly one value and then complains about garbage at the end.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-16 16:47:11 -07:00
Thiago Macieira 22eb998127 Refactor the parsing code to move the CBOR decoding to pure C
This will allow me to write a tool that does the pretty-printing of the
CBOR stream.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-16 15:34:01 -07:00
Thiago Macieira ed5b57c082 Fix warning that _BSD_SOURCE is redefined with a different value
Glibc's features.h defines it to value 1, so do the same.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-07 16:38:27 -07:00
Thiago Macieira 071815f378 Add a test that makes sure that the CBOR sources compile as C++
We'll need that in Qt and the Arduino AVR-GCC apparently does it too.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-07 16:37:49 -07:00
Thiago Macieira 521d729c02 Fix compilation on systems that don't have <arpa/inet.h>
Provided they have GCC 4.3 or a GCC-like compiler. For GCC, we were
already not using htonl and ntohl and for GCC >= 4.8, we weren't using
htons and ntohs either. So make it official and stop using for GCC 4.3
through 4.7, removing the #include too.

Systems without GCC will still attempt to include <arpa/inet.h>, as we
hope that ntohl/htonl are more optimised than a hand-rolled 32-bit byte-
swap.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-07 16:30:39 -07:00
Thiago Macieira 13c85791dc Fix compilation when UINTxx_MAX aren't defined
C99 also requires them in <stdint.h>, but some systems (Arduino 1.6.3)
are lacking the definitions.

Unlike for SIZE_MAX, we know the number of bits of the exact-width
integer types (C99 7.18.1.1), so we just define/use the values we need.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-07 16:30:39 -07:00
Thiago Macieira b5c9b25736 Fix compilation when SIZE_MAX isn't defined
Force it to be defined by using the well-defined conversion from signed
to unsigned.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-07 16:30:28 -07:00
Thiago Macieira 81f33434dd API Change: make cbor_encode_{float,double} take the value by-value
Instead of by pointer, which was inconsistent with all the other
cbor_encode_xxx functions.

Now only cbor_encode_half_float is an exception, but since the C
language doesn't have a standardised half-float type, we can't have it
in the API. Even for targets where GCC has an __fp16 type, we couldn't,
as the manual says (Top > C Extensions > Half-Precision):

  In addition, you cannot declare a function with a return value or
  parameters of type `__fp16'.

But I might remove the function and force you to use
cbor_encode_floating_point.

Fixes: #3
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-07-02 16:16:28 -07:00
Thiago Macieira ff130bc339 Split the two versions of cbor_value_{dup,copy}_strings
This introduces the checking of the actual type and also gets us type-
safety of the binary data.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-19 15:15:33 -07:00
Thiago Macieira fc87093649 Fix dup_string() using of an uninitialised value
Unlike calculate_string_length(), dup_string() failed to pre-initialise
*buflen to SIZE_MAX. I believe I had chosen not to call the calculate
function to make the code smaller, so the compiler won't have to move
the parameters around just to move them back to the same register in
order to call copy_string().

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-19 15:01:35 -07:00
Erich Keane 71ad491bdc Fixed inline function linking in C99
The CBOR_INLINE_API define previously was just 'inline'
for newer C compilers, however this was causing a failure
to link to these functions in the event that optimizations
were turned off.  This fix removes the condition on newer
compilers, and makes it ALWAYS static inline unless
the compiler is in C++ mode.

Signed-off-by: Erich Keane <erich.keane@intel.com>
2015-06-18 10:32:16 -07:00
Thiago Macieira 8f3fb7833b Fix using of my own assert()
The C standard says that if you #include <assert.h> again, it will
redefine assert() (because you my have defined or undefined NDEBUG).
Since that is the case, we need to "fix" assert as the last #include.

The "fix" has two advantages:
 1) it "uses" the condition, so you don't get a warning for variables
only used in asserts, like
cborparser.c:236:15: warning: unused variable ‘err’ [-Wunused-variable]

 2) it tells the compiler that the condition is always true in release
mode, so it may optimise the code for that case. In other words, bad
things might happen otherwise. There's currently no measured benefit in
code size for GCC 4.9 with -Os, but it does additionally suppress bogus warnings
from GCC:

cborparser.c:244:17: warning: ‘length’ may be used uninitialized in this function [-Wmaybe-uninitialized]

The compiler is saying that 'length' could be uninitialised if
extract_number returned CborErrorIllegalNumber. But since we asserted
that it isn't the case, all code paths that would use the uninitialised
value are eliminated. In debug mode, those code paths are eliminated by
assert() calling the noreturn function __assert_fail.

If the compiler had inlined extract_number, it would also have
eliminated the comparison additional_information > Value64Bit (line 92)
by dead code elimination.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 16:32:55 -07:00
Thiago Macieira ec94d5d215 Fix the build with non-GCC, non-Clang, non-ICC, non-Solaris, non-Windows
Oops, the #ifndef was missing the "n"

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 16:18:37 -07:00
Thiago Macieira 4540cd9054 Fix build with Clang: __clang__ does not contain the major version
It's actually in __clang_major__, but instead let's check that the
builtin intrinsics that we were looking for are actually present. This
should also fix any issues with Apple's backports of Clang into XCode.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 16:12:55 -07:00
Thiago Macieira f5cb94b522 Remove the inline marker from extract_{number,length}
Clang insists in inlining extract_number even with -Os, which increases
the code size considerably. Without the markers, now both Clang and GCC
inline extract_length but not extract_number.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 16:10:49 -07:00
Thiago Macieira 1a66e12c4c Simplify add_check_overflow's fallback
Unsigned additions are guaranteed to operate on modulo 2, so if there's
an overflow, the sum will be smaller than both v1 and v2. We don't need
to check both, though, one is enough.

Tested with GCC 4.9 to actually produce a jump-on-overflow instruction
on x86-64:

 108:   add    %r15,%r11
 10b:   jb     1cb <iterate_string_chunks+0x1cb>

With GCC 5:

 108:   add    %r9,%r11
 10b:   jb     123 <iterate_string_chunks+0x123>

With Clang:

  a9:   add    %rdx,%r14
  ac:   jb     1d1 <iterate_string_chunks+0x1d1>

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 16:09:26 -07:00
Thiago Macieira 1de31a4705 Invert the add_check_overflow fallback's return value
Match the intrinsic we were trying to use. This makes the check now work
on GCC 5 and Clang. Previously, it would always fail.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 16:01:16 -07:00
Thiago Macieira f4ba61e97b Fix compilation with older ICC and GCC
GCC didn't introduce __builtin_bswap16 until version 4.8. I don't know
which version of ICC introduced it, but _bswap16 seems to have been
supported for longer (a 2012 commit to include/linux/compiler-intel.h
used it).

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 15:36:10 -07:00
Thiago Macieira 5752ce5df3 Convert all the API using char for binary data to uint8_t
Now char is only used in the text string API, which is supposed to be
UTF-8.

Negative values for UTF-8 is stupid by itself, but the C++ committee
felt no need to add char8_t, so let's live with it.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 15:16:03 -07:00
Thiago Macieira f1c8d65416 Implement unreachable() for MSVC
That requires also defining likely() and unlikely().

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 12:08:54 -07:00
Thiago Macieira 5934a9f60c Improve the byteswapping functions with GCC and Clang
Force the code to use the __builtin_swapXX functions if we know them to
be available. GCC added them in version 4.3 and Clang in version 3.2.
But since Apple released XCode with a 3.2 version prior to the release,
we can't check for Clang version numbers, we have to use __has_builtin.

In order to force the using of the builtins, we can't use ntohl and
ntohs, so we define our own set of macros.

This fixes the issue that glibc's ntohs is implemented with a ROR
instruction instead of BSWAP or MOVBE. It also fixes the cross-build
using Clang because glibc apparently assumes that you either have GCC or
you have x86.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 11:55:28 -07:00
Thiago Macieira 05e37baea5 Fix two missing error condition detections in encodeVariant
One caused a test failure, the other caused memory corruption (valgrind
complained about jump conditions on uninitialised data).

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-16 11:08:36 -07:00
Thiago Macieira 774fad81c2 Work around qmake failing to parse tst_encoder.cpp
Move the #include for the moc file closer to the top so that qmake sees
it. Something in the source code is throwing qmake off and it can't
parse the dependencies.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2015-06-15 16:25:03 -07:00