Pascal Strings

2008-04-11

public static final char[] FILENOTFOUND = {'F','i','l','e',' ','n','o','t',' ','f','o','u','n','d'}; is missing!

2008-04-11

M.L.:
It would be nice to know the context of this code.
It may be a part of some kind simulation of abstract machine or processor where those constructs make much more sense.

Simulation of Brillant Paula's brain? Now it makes sense!

2008-04-11

Well if it had been written in C#, it would look much simpler because instead of

private final static char[] intDigits =
   {'0' , '1' , '2' , '3' , '4' , '5' ,
   '6' , '7' , '8' , '9' , 'a' , 'b' ,
   'c' , 'd' , 'e' , 'f' , 'g' , 'h' ,
   'i' , 'j' , 'k' , 'l' , 'm' , 'n' ,
   'o' , 'p' , 'q' , 'r' , 's' , 't' ,
   'u' , 'v' , 'w' , 'x' , 'y' , 'z' };

you can just write:

private const string intDigits = "0123456789abcdefghijklmnopqrstuvwxyz";

</Troll>

2008-04-12

Well, that's a blast from the past - writing this is probably a mistake (assuming that most people here want to "burn the witch") but I think it's worth putting some context in for posterity, it also helps explain what looks like insane code. (You can of course still disagree with the reasoning we used for implementing it).

The year: 2002/2003ish

Imagine a real-time distributed system, hundreds of machines, each with several processes on them. The processes talk to each other via socket connections using a highly optimised low-latency message passing library.

In general you could pass data around with this library (which did it in a binary format) with no problem at all as the data tended to be doubles and ints, but every now and then there was the need to pass strings around. The library happilly handled strings, but the receiving process would have to allocate memory for the incoming string. There was also a lot of wasted memory form int/double to string conversion.

Remember, this was a realtime system, and at the bleeding edge of JVM use (JRocket from memory). Garbage Collection ergonomics weren't as good as they are nowadays - this was a problem because if the GC kicked in, it stopped the world. Bad news for a real-time system.

So, passing primitives between processes was fine as there was practically no memory consumed by doing so (socket overheads aside).

The senior architect decided that we needed to find a way to stop wasting memory on transient strings and string manipulations. Hence PStrings were born. (passing char[]s around instead of Strings).

So, where are we:

Stop using Java Strings, start using char array backed 'pstrings' and keep a pool of suitable char arrays kicking around. Reuse the pre-existing char[]s as much as you can to avoid memory growth.
A need to convert doubles and ints into Pstrings, as you can no longer use Java's built in String converters/parsers (take a look, they're great, but they do need to allocate memory).

As I wrote the PString library you'd think I'd recognise isPString, but I don't. Looking at it here it looks plain wrong so I can only assume it has been modified since, or something is missing. Perhaps it's been refactored since originally developed and needs further cleaning up? I expect the JIT optimizes it down so it's not a performance hit, but the code's rubbish. I'm also surprised at the lack of comments, I tend to comment heavily, especially for something unclear.

All the other code in the WTF relates to number to string representation conversion, which you'll find dotted around the web and in Sun's Java source code. I used (with permission) code from the following book I believe: http://jlbtc.eduunix.cn/index/html/java/O'Reilly%20-%20Java.Performance.Tuning.2nd.Edition.eBook-LiB/0596003773_javapt2-chp-5-sect-3.html There were a couple of bugs in it which we helped fix, but as you can see, efficient double->string conversion is not a simple task, there are lots of look-up tables used to speed things up.

This appears to be someone else who has used the same library: http://www.koders.com/java/fid9B2B30D4084F831F061F3440EB40815111402793.aspx?s=mdef%3Ainsert

Finally, let's pose a few questions

Q. Are all of those lookup tables and crazy looking 'NaN' and 'Infinity' char[] wrong? A. No, that's how you do high-speed number to string conversion. For speed, look-up tables can be your friends whilst sacrificing memory.

Q. Are they worthy of a WTF A. Hell yeah. I scratched my head fixing a bug in the code from the book for quite a while. It would have taken me months to write one from scratch, and it wouldn't have been nearly as efficient.

Q. Are 'PStrings' a good idea? A. Back then, after repeated analysis with OptimizeIt and JProbe against our processes, I'd say it was the right thing to do. Nowadays, and having talked to Brian Goetz about modern day GCing, I'd say almost definitely not. JVMs have moved on and the GC is much improved nowadays.

Q. Am I sure it's my code? A. It might not be, but I definitely coined PStrings back in 2003 for our project, and it supported number<->string conversion. Let's be honest though, it's not exactly the most creatively named library (Hmm, pascal style strings, what could we call it?).

Cheers, Rob

Pascal Strings

Featured Comments