Random .NET fact: String.Join() throws a “fake” OutOfMemoryException

  • Date: October 9, 2016

The implementation of String.Join internally calculates the expected output length. If the calculated length is greater than max int32 (~2 billion characters), it will throw an “OutOfMemoryException” even though the system is not out of memory at that time.

It takes about 4GB to store 2 billion characters; since .NET stores strings in UTF-16 format, two bytes per character. However, .NET has an array memory limit of 2GB (unless on a 64-bit environment with gcAllowVeryLargeObjects enabled). As the code comment noted, this pre-check only catches the extreme case. The remaining of the method may still cause an OutOfMemory exception even if the pre-check passes.

For the record, .NET has an InsufficientMemoryException class for situations like this. However, this exception type was introduces in .NET 2.0. To stay backward compatible, the method remains throwing the OutOfMemoryException.

The following snippet comes from the official implementation of the method:

int jointLength = 0;
//Figure out the total length of the strings in value
int endIndex = startIndex + count - 1;
for (int stringToJoinIndex = startIndex; stringToJoinIndex <= endIndex; stringToJoinIndex++) {
    if (value[stringToJoinIndex] != null) {
        jointLength += value[stringToJoinIndex].Length;
    }
}

//Add enough room for the separator.
jointLength += (count - 1) * separator.Length;

// Note that we may not catch all overflows with this check, 
// since we could have wrapped around the 4gb range any number of times
// and landed back in the positive range.
// The input array might be modified from other threads, 
// so we have to do an overflow check before each append below anyway.
// Those overflows will get caught down there.
if ((jointLength < 0) || ((jointLength + 1) < 0) ) {
    throw new OutOfMemoryException();
}