I am not using the data in the other columns at the moment. That column is a string, I don't really like how it doesn't clearly indicate it's going to write to the following 5 memory locations when it reads a string.
I see the INTs are 16 bit and not 8 bit and it is stored little endian that was throwing me for a loop a little bit.
I read up on the documentation on the direct read, and I was really underwhelmed with the detail in it. I was hoping to use it to store some graphical data. The one I linked to above is still the best resource I've found on direct read and it still seems lacking.
The problem turned out to the the ML was storing the 8bit values in each Long. So it was using up the 6 character size with a bunch of nulls. GOOD FOR NUMBERS, BAD FOR STRINGS!