Character Encoding in API Responses

August stores all user-facing strings in an HTML-escaped (safe) way. The goal is to prevent malicious inputs from users from affecting client applications.

Here is an extreme example of encoding a LockName like ©®`~|•^'#&;

curl -X GET -H "x-august-api-key: $APIKEY" -H "x-august-access-token: $AUGTOKEN" -H "content-type: application-json" https://api-production.august.com/users/locks/mine

{"ABC....123LOCK ID": {
        "HouseID": "def...-...-123HOUSE ID",
        "HouseName": "Andy's Desk Prod 2",
        "LockName": "©®\\`~|•^'#&;",
        "UserType": "superuser",
        "macAddress": "mm.aa.cc.aa.dd.rr"
    }
}

FAQ

Ok, so how should I unescape these strings? you might ask.

Android

On Android August uses unbescape for encoding and decoding.

http://www.unbescape.org/

unbescape is a Java library aimed at performing fully-featured and high-performance escape and unescape operations for:

• HTML (HTML5 and HTML 4)
• XML (XML 1.0 and XML 1.1)
• JavaScript
• JSON
• URI / URL (both paths and query parameters)
• CSS (both identifiers and string literals)
• CSV (Comma-Separated Values)
• Java literals
• Java .properties files (both keys and values)

unbescape is Open Source Software, and it is distributed under the terms of the Apache License 2.0.

iOS

On iOS we use this to decode HTML entities (based on http://stackoverflow.com/a/1453142 ):

+ (NSString *)stringByDecodingXMLEntities:(NSString *)stringToDecode {
    if ((stringToDecode == nil) || (![stringToDecode.class isSubclassOfClass:[NSString class]])) {
        return nil;
    }
    NSUInteger myLength = [stringToDecode length];
    NSUInteger ampIndex = [stringToDecode rangeOfString:@"&" options:NSLiteralSearch].location;

    // Short-circuit if there are no ampersands.
    if (ampIndex == NSNotFound) {
        return stringToDecode;
    }
    // Make result string with some extra capacity.
    NSMutableString *result = [NSMutableString stringWithCapacity:(myLength * 1.25)];

    // First iteration doesn't need to scan to & since we did that already, but for code simplicity's sake we'll do it again with the scanner.
    NSScanner *scanner = [NSScanner scannerWithString:stringToDecode];

    [scanner setCharactersToBeSkipped:nil];

    NSCharacterSet *boundaryCharacterSet = [NSCharacterSet characterSetWithCharactersInString:@" \t\n\r;"];

    do {
        // Scan up to the next entity or the end of the string.
        NSString *nonEntityString;
        if ([scanner scanUpToString:@"&" intoString:&nonEntityString]) {
            [result appendString:nonEntityString];
        }
        if ([scanner isAtEnd]) {
            goto finish;
        }
        // Scan either a HTML or numeric character entity reference.
        if ([scanner scanString:@"&" intoString:NULL])
            [result appendString:@"&"];
        else if ([scanner scanString:@"'" intoString:NULL])
            [result appendString:@"'"];
        else if ([scanner scanString:@""" intoString:NULL])
            [result appendString:@"\""];
        else if ([scanner scanString:@"<" intoString:NULL])
            [result appendString:@"<"];
        else if ([scanner scanString:@"&gt;" intoString:NULL])
            [result appendString:@">"];
        else if ([scanner scanString:@"&#" intoString:NULL]) {
            BOOL gotNumber;
            unsigned charCode;
            NSString *xForHex = @"";

            // Is it hex or decimal?
            if ([scanner scanString:@"x" intoString:&xForHex]) {
                gotNumber = [scanner scanHexInt:&charCode];
            }
            else {
                gotNumber = [scanner scanInt:(int*)&charCode];
            }

            if (gotNumber) {
                [result appendFormat:@"%C", (unichar)charCode];

                [scanner scanString:@";" intoString:NULL];
            }
            else {
                NSString *unknownEntity = @"";

                [scanner scanUpToCharactersFromSet:boundaryCharacterSet intoString:&unknownEntity];


                [result appendFormat:@"&#%@%@", xForHex, unknownEntity];

                //[scanner scanUpToString:@";" intoString:&unknownEntity];
                //[result appendFormat:@"&#%@%@;", xForHex, unknownEntity];
                NSLog(@"Expected numeric character entity but got &#%@%@;", xForHex, unknownEntity);

            }

        }
        else {
            NSString *amp;

            [scanner scanString:@"&" intoString:&amp];  //an isolated & symbol
            [result appendString:amp];

            /*
             NSString *unknownEntity = @"";
             [scanner scanUpToString:@";" intoString:&unknownEntity];
             NSString *semicolon = @"";
             [scanner scanString:@";" intoString:&semicolon];
             [result appendFormat:@"%@%@", unknownEntity, semicolon];
             NSLog(@"Unsupported XML character entity %@%@", unknownEntity, semicolon);
             */
        }

    }
    while (![scanner isAtEnd]);

finish:
    return result;
}