ISO: avoid undefined behavior and integer overflow in the fletcher checksum calculation
The fletcher checksum calculation would sometimes left-shift
a negative number, which is an undefined operation. Rework the
code to avoid this.
checksum.c:186:20: runtime error: left shift of negative value -36
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior checksum.c:186:20
Unlike some checksum routines that use the defined semantics of
2's-complement unsigned overflow to their advantage, this one
gets the wrong value if it is allowed to overflow, due to the
use of mod-255.
Convert c1 to uint64_t to avoid overflow.
checksum.c:163:16: runtime error: unsigned integer overflow: NNN + NNN
cannot be represented in type 'unsigned int'
Use integers during subtraction to avoid implicit conversion to unsigned
when calculating both x and y
checksum.c:172:18: runtime error: unsigned integer overflow: NNN - NNN
cannot be represented in type 'unsigned int'
checksum.c:172:9: runtime error: implicit conversion from type
'unsigned int' of value NNN (32-bit, unsigned) to type 'int' changed
the value to -NNN (32-bit, signed)
checksum.c:173:12: runtime error: unsigned integer overflow: NNN - NNN
cannot be represented in type 'unsigned int'
checksum.c:173:9: runtime error: implicit conversion from type
'unsigned int' of value NNN (32-bit, unsigned) to type 'int' changed
the value to -NNN (32-bit, signed)
(backported from commit
c5b54bfbd68b03f7997feaa277db30d399975a4d)