[PATCH] avoid undefined behavior in fmt_scaled()

Nickolai Zeldovich nickolai at csail.mit.edu
Wed Jan 9 04:36:22 EST 2013


fmt_scaled() computes the absolute value of its argument, and returns an 
error if the result is still negative (i.e., when the value is LLONG_MIN). 
However, taking the negative of LLONG_MIN is undefined behavior in C, and 
some compilers (e.g., gcc-4.7.2) will optimize away the 'abval < 0' check 
as being always false (since it can only be true with undefined behavior).
The patch below performs an equivalent check without invoking undefined 
behavior.

Nickolai.

--- openssh-6.1p1/openbsd-compat/fmt_scaled.c	2008-05-19 18:57:08.000000000 -0400
+++ openssh-6.1p1/openbsd-compat/fmt_scaled.c	2013-01-08 12:18:29.883527421 -0500
@@ -196,12 +196,16 @@
  	unsigned int i;
  	unit_type unit = NONE;

+	/* Not every negative long long has a positive representation. */
+	if (number == LLONG_MIN) {
+		errno = ERANGE;
+		return -1;
+	}
+
  	abval = (number < 0LL) ? -number : number;	/* no long long_abs yet */

-	/* Not every negative long long has a positive representation.
-	 * Also check for numbers that are just too darned big to format
-	 */
-	if (abval < 0 || abval / 1024 >= scale_factors[SCALE_LENGTH-1]) {
+	/* Check for numbers that are just too darned big to format. */
+	if (abval / 1024 >= scale_factors[SCALE_LENGTH-1]) {
  		errno = ERANGE;
  		return -1;
  	}


More information about the openssh-unix-dev mailing list