Apparently someone wanted to be the first person ever to trade oil at $100 so he bid it up, bought it and immediately sold it. Didn’t even cost him that much. He just wanted to have been the first and that was it.

Should you take it?

Do you like money?

Do you like money more than anything else in the world?

That’s the question you have to ask yourself.  You’ll make more than enough to live on.  You’ll probably get a bonus ranging from a couple months salary to the type of money your parents wouldn’t be comfortable thinking about.  The office will be wonderfully furnished, and the work will be interesting.  They’ll only hire the hottest HR women, the prettiest secretaries.  Free drinks, snacks, bar after hours, a car service, free insurance, paid breakfast lunch and dinner.  It’s a crazy world.

It’s also all you’ll have.

Traders – you’re in it for money.  That’s what you *do* after all.  Play along for a couple of years, save your bonuses, and start a restaurant.  Live a life of boring luxury if you want.  IT folks, you’re my people.  I don’t do it for the money – it’s sure as heck nice though.  You probably do it because it’s coolest work you’ll ever find.  You can use your beloved language (ahem, perl) because it’s what you’re the fastest and most productive in. The best, fastest, newest everything you can find.  But you’re working for the traders.  The traders make the company money, you are an expense.  Sure, they give lip service saying that you’re all equal, but you’re not, are you?  When they stay late so they can have more profits on their books it’s a measurable thing that shows up at the end of the year in their bonus.  When you stay late, it’s so you don’t get yelled at.  Uptime doesn’t measure to actual dollars.

Why you should say yes

• It pays well!

There’s no place in the world like the financial industry.  The companies nearly literally do nothing but create money.  Salaries are good, bonuses are better.  I don’t know of anyplace in the world you can make as much.  Period.

• Benefits

Benefits match salaries. 100% paid health, vision, dental, AD&D, life insurance policies.  Free food.  Paid transportation – public, light rail, parking passes.  Drinks and frequent outings and parties.  Snacks and gym memberships.

• It’s interesting work

Figuring out low latency systems, writing programs that handle billions of dollars, tying systems made of dozens or hundreds of smaller programs together.  New algorithms, different languages, the flexibility to change things.  All make for time flying doing cool things.

• Smart people

The smartest people money can buy!  They’re not hidden in a think tank or back room either – they’re right next to you making your brilliant ideas sound silly and trivial in comparison.  So many great people to learn from.  Steal their book lists, read their favorite websites, pick their brains.  Making you smarter makes the company money too, after all.

• Office environment

Ever seen a trader’s station?  The only limiting factor in the number of screens they get is their peripheral vision.  Six 30″ lcd’s orientated vertically with 19″ monitors on top of that.  Tech people get the hand-me-downs, of course, but you still get rather good hardware usually.  Aeron chairs, anything ergonomic the office manager can find.  Shiny toys bought by vendors for the company.  Giant-screen TV’s everywhere.  Kitchen to hang out in.  Everything is beautiful and functional.

• Cool toys

Brand new routers, switches, servers.  Fiber.  Network cards with their own operating system.  Fiber fabrics that use different frequencies of light to get more bandwidth.  GPS time servers.  At least gigabit everything.  Terabyte storage arrays.  Entire floors of a building designed around cooling.  Datacenters.  It’s really freaking cool.

Why you should say no

• You’re admitting you’re a whore

There’s no nicer way to put this one.  You are saying that  you value money over anything else.  Health, morality, family.  You’re only in it for the money, and the company you work for doesn’t do anything but make money.

• You will burn out

Ten hour days will get you fired for not showing up.  The pace is frantic, at the end of the day your brain melts.  Adrenaline from the awesome parts of the job doesn’t do it after a while.  Sleep is for poor people.

• The work stops being interesting

Yes, you can spend another year working to make your system trade faster.  Just like last year.  There isn’t even a target to shoot for.  Just “faster”.

• Smart, but narrowly focused people

The mighty Wolfram is famous for this.  “I’ll see you this weekend” “No, I’m watching the super bowl.”  “What’s a super bowl?”  The people around you (and you, by extension) focus so much on the ways to make the company more money that nothing else exists.  You have to be so good at what you do to stay ahead that that’s all you can focus on.

• Work-life balance does not exist

See also You will burn out, above.  They buy you for the hefty fees you are making.  Wake up early to get in before market open, work, then study your job when you get home.  We were actually planning to put convertible couches into the conference rooms for sleeping.

• Chaotic work environment

Ever been to a casino?  That’s a lot like the floor.  You don’t even have the token courtesy of cubicle walls to stop the ringing, talking, yelling, sheep noises, etc.  Add to that constant interruptions and a lot of work gets done.  Badly.

• Paranoia

These companies are secretive, and for good reason.  Everything from hardware, to network paths, to what programming language is fodder for the enemy.  Trading is a zero-sum game, after all.

Is it worth it?

Hell yes.  Or maybe not.  Try it.  If it doesn’t go against your ethics, you don’t mind taking a few years off from productive life, and can keep yourself broad enough while making it in the world there, it’s an amazing experience.  And the money doesn’t hurt.

At least if you can handle the negatives.  And they’re real.  You’re supporting traders.  You can be the head of IT, the lead SysAdmin, the brilliant network engineer.  Whatever.  You’re a monkey who works for the lowest trader.  If the company talks about work-life balance, it’s because they’ve had trouble with people quitting recently and that’s what they mentioned.  If they don’t, it’s because they don’t believe in it.  That said, it’s amazing work.  It’s fast work.  It’s educating work.  It’s inspiring and depressing.  If you can disassociate yourself from what you do, and be awesome at doing it, it’s really really worth it.  Personally?  I have no problems admitting I do what I do for money if the job is interesting enough that I’d do it anyway.  Give more to charity if it bothers you.  If you can’t handle time time, though, you’re not going to do well.  All I can do is try to present it so you can make an educated choice.

Proximity

For near exchange work, it’s all about proximity.  If you can get a cross connect to an exchange,  you want to be as physically close to them as you can.  In the same datacenter is a minimum.  On the same floor.  In the same area.  As physically close as possible.  On a trip to a datacenter, one of the jobs is scoping out what’s going on.  That new expansion is where the exchange is going?  You should be able to get access to pick your spot.  All that info is secret, of course, but that doesn’t matter.  You know where the exchange is going, that giant cage with custom power and cooling next to it?  That’s Citadel, I heard.  But you can’t compete with Citadel that way, but you can be the next best spot.

This is the area where you’re measuring in microseconds.   The differences here can also be made up in other ways.  Matching your routers to the exchange you’re connecting to.  Will a Cisco or a Juniper have better time?  Tune the algorithm.  Measure, measure, measure.

If you can’t get a cross connect directly, or you have lines coming in from elsewhere, your line makes a trip to the “meet me room” or some variation on the theme.  That’s where all the customers of the datacenter have their patches (that don’t go directly to someone else), and get connected to other folks.  Your outside line, your other cages maybe, exchanges, whoever it is you have to talk to.  Then somewhere in your cage or cabinet you have a patch panel where you can finally connect it to your equipment.

Of course, sometimes you get buildings like where the TSX floor is located.  You can’t just connect your equipment on one floor to another floor – that would be too easy.  You have to connect at your floor, it goes all the way down to the meet me room in the basement, and then all the way back up to the other location you’re connecting to.  All at a fee, of course.

The datacenter itself

The datacenters themselves are amazing in almost every way.  The way they’re not amazing?  Appearances from the outside.  Unmarked.  Actively nondescript.  Could be a warehouse, could be an office building.  Some  are multiple-block-long buildings out in New Jersey.  Some are closets in skyscrapers.  Some are multiple story former printing press buildings.  Some you can almost throw a stone to one of the busiest roads in the world.

One has a great view of the Statue of Liberty.

There tend to be similarities, though.  There are rows and rows of either racks, cages, or a combination of the two.  There’s alternating hot and cold aisles.  Cold air is forced either through the floor or from above via vents into the cold aisle where the servers draw it in.  The hot aisles collect the output from both sides and either take it in or let it go naturally up into an open space.  There are overhead ladders with network wiring that then breaks off and goes into the desired cages.  The wiring for the power is the same story as the cold air.  Either above or below.

The biggest limiting factor?  Power and a/c.  You can usually get space, and with that space comes a little bit of power.  How much?  Between 15 and 30 amps for the whole cabinet.  A 42 U space with between 8  and 18 or so used.  Hope you’re using 220v power for that extra efficiency!

Working in a datacenter is, like most things in tech and trading, an exercise in contradictions.  They’re super cold so you’re freezing, but if you’re moving equipment, you’re still sweating.  They’re also humidity controlled, so you’re usually thirsty – but there’s absolutely no liquid allowed inside.  You’re probably hooked up to one of the most advanced networks in the world, but getting online is a pain.  Wifi is strictly prohibited except for very temporary setups for working.  You brought your laptop, but there’s no place to plug it in!

That’s legit – no place to plug it in.  The cage has a PDU designed for use in cages like this, and with servers and networking equipment.  Like this:  If you didn’t have the foresight to order an adapter and/or your laptop won’t work on the voltage your cage is at –  then you have no power!

There’s also no food allowed, but there’s usually a break room where people stash goodies for later.  Bring drinks for there too, and label your bag.  The other problem is snack food does not a meal make, and they usually (on purpose) make these things as far from civilization as possible!  Usually there’s a small group working at the same time, so when you’re not going flat out as fast as you can, you’re sitting and waiting to be useful again.  Of course, after a few trips out you generally just start sneaking candy and drinks in anyway.  “Rules” eh?

If you’re lucky enough to have a whole cage of whatever size, you don’t have enough power to fill it up anyway.  So make yourself a work area.  Get a folding table and chair, keep tools around, have all the lengths and colors of cabling you use on hand, have a usable power strip and a couple extra ports.  It’s like a cold, loud home away from home!

Latency

When working with two or more exchanges, it’s more about latency than proximity.   Verizon guarantees 8.5 milliseconds per 1,000 miles – but that’s as the wire lays.    It’s 710 miles from Chicago to New York.  What time will your packet make the round trip?  Well, if you have a good line, 18ms is pretty good.  Less than that and you can make a business out of it. These folks for example advertises 17.2ms.  There’s rumors about 16.8 and less.  I want to know what the bandwidth along I-80 is.

When in a group of industry folks, someone will talk about a new path to NJ they’ve found.  “Does it follow I-80?” “Better!”  But you won’t get anything else out of them.

Within metropolitan areas, you still have to find the best paths between places.  A few hundred microseconds are too many to waste.

What is  dark fiber exactly?  Well, it’s a piece of fiber optic cable that you connect to your equipment at both ends.  It’s literally dark until you “light” it up. There’s no telephone company routers in between to change the pathing or add latency.

Measure twice.  Actually, just keep measuring.

Everything – everything everything depends on your measurements.  When did we send it, when do they say they got it, when did we get it back.  Does that all match what the router says?

I used to ask family members what the most important part of the measuring operations was.  No-one guessed knowing the time.

You have got to have a good time infrastructure.  NTP is okay, but where are you getting your TP from?  The internet is way too slow to be accurate enough for what we’re measuring here.  You need a network time appliance in every metropolitan area.  Every single datacenter would be awesome.  Meinberg is the gold standard.  It’s cool technology.  The other problem with NTP is it’s not accurate enough in practice (in theory it is, but that’s a nice theory).  PTP.  How accurate is accurate enough?  When you’re measuring the time it takes for an update to get into your decision engine until the time it takes to make a decision to measure how good your software is, you need in the nanos.  I’m not saying single-digit nanosecond resolution.  (Yet)

And throughout the day, things can change.  Your monitoring infrastructure has to keep up not only with the hardware running your network, but the Quality of the Service on it, too.  Things happen.  Fiber gets cut.  The telco re-routes you without any notice. Suddenly you’re getting picked off because you’re a millisecond slower than everyone else.

It’s better to drop out of the market and take the opportunity cost than to be in and making other people money.

Look up haircut sometime if you want to know how it all relates to how much you can leverage.

So, your sponsor takes on the risk of you trading somewhat in their name.  At the end of the day, they have a list of what you traded, and you have a list of what you traded.  The folks in mid-office then have to deal with your clearing firm and your sponsor so everyone agrees just what happened when the traders were out playing all day.  The clearing firm is the go-between the two parties of the trade.  Basically, they’re a really valuable lubricant in the wheels of the market.  So GS gets the list from your clearing firm, you send what you think you traded, and hopefully you don’t get a call from them asking where the 10,000 shares of Berkshire are in your list.

Clearing – in practice

It’s fairly simple actually.  You get a method by which you send  your report.  Let’s say FTP.  You upload it with a certain filename format to a server with a given username and password.  Easy.  The hard part is generating the file.  You could have one database, you could have a hundred.  There could be one format, or dozens.  You could have to download and parse text files.  And don’t forget versioning either.  So you download all the executions you made.  Then you have to group them by exchange, symbol, and side, and add them up and average them out. So your file goes from, say, 200,000 shares of AAPL traded to three lines:

AAPL,B,100203, 190.91,blah,blah
AAPL,S,101238,191.33,blah,blah
AAPL,SS,2231,191.02,blah,blah

You combine all of them together for that particular exchange, in that particular format, and upload it to  your firm.  They compare it with their (much simpler to generate, because they only have one potential input) file and if there’s a problem they email and/or call.  It’s a surprisingly simple program in perl.  Collect your data from all the different sources, put it into a huge hash, print it out to a file, upload it.  Put in error checking and notification.  Do this for every exchange you trade on, at the specified time(s) daily.

Only two important things to note:

If that file is wrong or doesn’t get uploaded – they don’t trade the next day until it’s fixed.

“The next financial crisis will be caused by a divide-by-zero error in someone’s perl script” (Citation needed)

Compliance

A huge amount of time and money is spent on compliance at these firms.  And at the same time it’s a total afterthought.  The traders rule the roost, and they don’t care about it –  until they can’t trade anymore or their bonus is lower because of a failed audit.  Essentially here’s what it comes down to.  You have to keep track of every trade and every order you make, and keep it forever.  Sure, they say seven years, or five years, but it’s forever.  For one – seven years really is forever in computer terms.  Creating something that’s archivable over that long is essentially creating it forever.  And for two – there’s nothing saying that a year from now they won’t want the records for 10 years, or 15 years.

So you’re keeping track of every order and every trade.  While you’re doing it, you have to keep your positions at the time if at all possible.   Why?  Shorting stocks for example.  Traders are allowed to short, but it goes against the haircut and reduces the amount they can trade.  If it’s on the easy-to-borrow list though, it’s a different story.  Anyway, the regulators come in and you have to prove that you weren’t shorting a stock.  Which means you have to know or be able to derive your position at that time.  Then you have to prove that at the millisecond you placed the order, it wouldn’t be a short.  Which also means accurate time across the board of your systems.  It’s fun, see?

The easy to borrow list is exactly like it sounds.  A list of things that are easy to borrow.  Some stocks are so available to trade – so liquid – that they get on a special list that doesn’t really count as short selling! Link

In addition to ordering what and when, you also have to be able to prove who.  So every trader has to have a unique login so you can prove who did what.   If Bob over in the corner is doing something illegal or unethical, you have to be able to prove it was bob doing it.  That way when he’s caught he’ll just take himself down and not the company.  So you have to record every method by which someone can communicate with evil peoples.  You have to keep (like I said, pretty much forever):

• Email
• Chat
• Phone
• Tin can and string
• Any other electronic medium

Funny story – one time the network went down for some reason.  Most places use IP phones now, which use the network.  Just to be safe, we had to call the exchanges to cancel our orders – but no phones!  So we all whip out our cell phones and call that way.  1- You can’t do that since they’re not logged and monitored. and 2- “Does anyone have something other than AT&T?  I can’t call anyone!”

Email has to go through your email servers and be backed up, instant messaging has to go through a proxy that logs all conversation, phones (at least for traders) have to keep recordings, etc etc.  One of my better scripts was actually to do a daily dump of chat logs, bundle them per user, and send it as an email to a special email address.  Killing two birds with one stone!   Regulators will seriously come in and say “we want every communication trader X made from this date to this date.”  And you say “okay” or you get a fine.  They also do spot-checks at least once per year.  “Give us everything you traded from this date to this date.”  And you say “okay” or you get a fine.  Incidentally, if you want to see roughly what the code for chat log retention looked like:

#!/usr/bin/perl
use strict;use warnings;
use include;
my $ch='chat_host';
my $cu='chat_user';
my $cp='chat_pass';
my @worries=();
my $to='[email protected]';
my $today=&include::getdate('YYYY-MM-DD');
my @senders = &include::sql($ch,$cu,$cp,"select distinct(senderid) from message_log where date(modified)='$today'")
  or &freakout($!);
foreach my $sender(@senders)
{
  my @lines=&include::sql($ch,$cu,$cp,"select msg_text from message_log where date(modified)='$today' and senderid='$sender'");
  if(!scalar(@lines))
  {
    push(@worries,"No lines for $sender");
    next;
  }
  &include::send_email($to,join("_",split(" ",$sender)).'@backuphost.mycompany.com','Chat logs for $today',join("\n",@lines))
    or push(@worries,$!);
}
if(scalar(@worries))
{
  &include::send_email('[email protected]','[email protected]','Possible problems with chat backups',join("\n",@worries));
}
sub freakout
{
  my $wtf=shift;
  &include::send_email('[email protected]','[email protected]','Error with chat logs for $today',$wtf);
  exit(1);
}

It’s a  simplified but only because the hard stuff is hidden in my include module I keep around to make my life easier – it was(is) literally this short.  We traded every single day and there was chat (and chatbots) every day, so if there wasn’t anything in the logs there was definitely a problem.  In plain language:

Do some housekeeping (includes, variable setup for hostnames and the like, the date, etc)
Get a list of everyone who sent an IM that day and if there's a problem, send a freakout email with what happened.
For every person in that list, send an email with their name as the "from" address and the chats they sent as the body
  (if there's a problem, add it to the list of worrying things)
If anything is worrying, send an email to that effect to me.

The reason this is so simple is kind of a lucky coincidence.  The chat logs are in one table, the ID is the name of the person, and the text of the log includes the time and who it’s to.  If it didn’t, I’d have to have another line to get the identity of the person – no big deal.  I’d also have to have a temporary array of lines that I’d append the “to”, “time”, and “text” to as one entry, then send the email with the contents of that array instead.

Market Making and Market Taking

The individual markets want action.  They want you to be able to buy or sell easily on their markets.  Sure, they go for big numbers, too, but what they want is liquidity.  If average Joe on the street can’t go into BillyBob Exchange, LLC and buy 100 shares of That Gargle Interweb thing, but can elsewhere, they’re not going to be an exchange for much longer.

MarketMakers add liquidity to the market

There’s not always enough people in the market at a given time to make it liquid enough for every single instrument traded on it.  So the exchanges have people called MarketMakers.  Their job is literally that – to make the market.  If you’re a registered marketmaker, you have to be in the market for a given amount of time or get in trouble.   You can be a marketmaker for one symbol or a hundred, it’s whatever you can keep up the action for.  Marketmakers can make an okay living doing this, but it can be risky too.  They live on both sides of the “price” for a symbol.  If something is “worth” some arbitrary amount of money, they will buy it for a little less than that, and sell it for a little more.  (Bid and Ask)  If the market moves too quickly in one direction, there’s a risk that they are making bad trades because they haven’t updated their prices to account for the risk.  More risk means trading wider around the price.  You can also adjust your price to account for your position.  If you’re long something, you may be willing to take slightly less money to unload so you don’t have that position.  In return for this risk – the exchanges “bribe” the marketmakers.  They get a nice discount on their trades.

While it may cost you $7.95 per trade, a marketmaker can pay 1/1000 of that – or less.  Or nothing.  Or they can get paid to trade.

It’s in the ratio.  Quoting (marketmaking) gets you a rebate, taking liquidity costs you money.  Marketmakers also have an obligation to quote a certain amount and hours per day.  That doesn’t mean they can’t quote extra wide, but they’re not making money then and not making the exchange happy.

Arbitrage

I love arbitrage.   In it’s most basic definition, it is taking advantage of price differentials.  Say the price of AAPL on the TSX is $201/share, on Nasdaq it’s $200 a share.  So you buy it on Nasdaq and sell it on the TSX and pocket the dollar, minus any fees.  My favorite is Berkshire Hathaway (BRK) – it has two classes of stock, A and B.  They’re related – you can exchange A for B.  One class-A share equals 30 class-B shares.  Since most people don’t buy BRK to vote, there is a very, very strong relationship between the two.  So if it’s ever not very very close to a 30:1 price ratio, there’s a simple arbitrage opportunity.

A surprising amount of what the proprietary trading companies do is arbritage.  Everything from distance (latency, really) based, to ratios like the BRK example, to the symbols within a larger context.  For example, you can trade the component stocks of the S&P 500 against a future of the S&P 500 index itself.  Since one is literally made of the other, the price in one underlying symbol means that the other is worth a different amount.

Another good example is pricing options.  Most of the market prices options based on the Black-Scholes formula (roughly speaking).  This means that there’s a pretty standard way of figuring the prices that the market generally gravitates toward.  Those who can program a faster algorithm for computing this can price them faster, and have a pure arbitrage opportunity.  Options are also fun in another way.  You can “create” shares from options.  Wikipedia explains it better than I could, but basically you can buy and sell options in a certain way that lets you simulate having a share of stock.  Or better yet, simulate selling one or selling one short when you can’t really sell it short.

Correlation

This is basically taking advantage of similarities between companies.  If oil goes up, companies that use lots of oil tend to go down.  Or on a more practical example I like to give, if oil goes up, American car companies tend to go down.  Hand in hand with that, similar companies tend to follow each other also.  GM and Chrysler did for a while.  Big steel companies, when they existed.  Heck, weather futures against crop futures.  Think of a few on  your own, it’s fun!  Basically anything you can think of that may have an effect on something else can be priced.  It doesn’t have to be a 1:1 ratio either.  The companies aren’t priced the same, nor is the relationship between them perfect.

In practice – order types (list)

The market open and market close are where almost all of the fun in the markets happens.  Open especially.  You can saturate a 1GB connection to a market just with data feeds.  (Get a bigger connection).  For perspective, that line that is getting completely overrun with data is probably 500 to 1000 times faster than yours at home. You put your quotes and orders in before open or right at open and the insanity happens.  Now when the market moves on you, you need to change your quotes.  If the “value” is $10.00 and you’re at $9.95 and $10.05 and it inches higher, pretty soon your asking price is so low that you’re losing money on every trade.  So you update the price to move with the market.  You can have one order in at the beginning of the day and update it 100,000 times throughout the day.  These orders (market or limit) are normally “day” orders, and are valid until canceled.  So you can update it or do a cancel and replace.  The are liquidity *adding* orders.  The marketmaker uses them to get into the market and stay there until filled, at which point they’ll adjust their prices depending on whether they’re long or short and put in a new order.  Why is this *adding* liquidity?  Because it is giving others the opportunity to trade.  You you just out there saying “Here I am, I’ll trade with you if you want. Here are my prices.”

Immediate or cancel, fill or kill, whatever you want to call it – these are the market *taking* orders.  You see an opportunity for some quick profit that will remove liquidity from the market.  There’s a price mis-match somewhere, someone put in an order you think is “wrong”, there are plenty of reasons.  But you only only only want to get it at that price.  If it’s not immediately filled at that price, the order is canceled and nothing happened.  Why does this remove liquidity? Because unlike the day orders, you’re not actually giving an opportunity to someone else to trade, you’re trying to take it away from someone else.   It removes one open order from the market, takes the numbers a little lower from the exchange, and thus they charge you for the privilege.

Another interesting thing about the markets that one has to account for is what’s called an Iceberg.  Say you have a lot of stock you have to sell.  Putting a giant order out there will change the market – people will see you need to sell a lot and the price will got down accordingly.   Of course, you don’t want the price to go down as much – you want more money for your stock!  The markets let you put something called an iceberg order into the market.  It lets you specify how much the market sees of the order.  Instead of 10,000 or 100,000 maybe you want to show 1,000.  Eventually the market will figure out that there’s a lot of selling going on and price itself accordingly, but you haven’t thrown it for a loop with your giant numbers – it’s gradual.  So you can set  your order as an iceberg, and the people on the other side ordering against you can also place their order for *more* than it shows the offer is.  So you see an offer in the market for 500 shares @ $10.00, and you think that’s a great price – you can try an order larger than that, and if it’s an iceberg, you’ll trade for the larger quantity.  Neat stuff.

Trading, a super short history

In the beginning, there was the marketplace.  And it was… primitive.  Multiple places in Europe started different kinds of exchanges to trade things.  Shares in companies, government securities, tulip futures (famously), etc.  For hundreds of years, it was open outcry at exchanges.  People would literally huddle in groups for each thing being traded and make deals in person.  They would write down what happened for the deal on pieces of paper and reconcile at the end of trading.  Phone lines elsewhere on the floor were added, runners would go to their brokers after getting a call from someone else who saw the price on the ticket, etc.  NASDAQ came around and was electronic.  It revolutionized the industry.  Other exchanges slowly came around until now nearly everything is electronic.

The rise of computers

Computers changed quite a few things.  Speed and reliability of execution were improved by an incredible degree.  Costs were lowered.  Barriers to entry removed.

Seats on the CME start at $750,000 for example link.  Nasdaq?  $2000 application fee and montly fees for membership ($3500 or so). Link

Per trade fees are also lower than they used to be.  Combined with networking advances, it’s the perfect recipe for high frequency trading.

Enabling factors

There are a few things that have to happen for this to work.  A firm wanting to get involved in high frequency trading needs kind of a perfect storm to make it in the market.

Fast data feeds from one or more markets.

The feed is a stream of data coming from the exchange listing what is happening.  People are bidding on products, people are offering products for sale, and sales are occuring.  The feed tells you all of them.

Fast connections back to the matching engine.

The matching engine is what it sounds like.  It matches up the buyer with the seller for a given product at a given price and sends that data back via the feed

Computer power to actually figure out if there’s a profitable trade to be made.

Companies usually locate their servers as close as physically possible to one or more exchanges.  The datacenters themselves are amazing.

Cheap trading.

(talked about more later).

So this all means?

Computers take in the feed, which is just what is going on in the market, and decide to make trades.  It’s fast.  How fast?  Less than 200 microseconds to make a decision and send it back out is completely reasonable.  Two hundred millionths of a second.  And it has to do a lot of other things.  If you are long (own) too many, you want to make it slightly less likely you’ll buy so you don’t end up with too many at the end of the day.  If you’re short(have sold more than you have), you have to track all of that.  High frequency trading is, quite simply, trading that’s done by computers, and only incidentally are humans involved.

We (the people) think of the strategies, turn them into parameters the computers can understand, test them, and then let the computers run them with someone watching.  Usually, someone spots or thinks they spot some advantageous idea and goes from there.  Some people will write it into the programming itself – meaning that the program that does the trading itself is re-written to account for the new idea.  Others will let the program modify itself – the people thinking up the ideas will enter programming language-like-code that then gets run by the trading program.

The rest of the posts will be more about specific areas I thought were interesting when I was there – or areas I was involved in a lot.

This is a series of posts about my experience as a sysadmin and programmer in the world of High Frequency Trading.  I spent two years intimately involved in the daily operations and planning of a player in the field.  My programs were (and are to this day) responsible for clearing, backups, logging, statistical analysis, and a hundred other things.  One handled/handles billions of dollars of trades daily.  I’ve been to datacenters, been to sporting events with brokers, and called exchanges to cancel all of our orders.  The people, the environment, the companies, and the competition are all aggressive and brilliant and exhausting.  There’s nothing like it in the world, I’m glad I was there, and I’m glad I’m gone. The purpose of this is to explain to family and friends what it is I was doing at this secretive company for all that time. If other people learn a bit about how it works, it’s extra credit.  It’s all as I remember having been out for three months – enough time to get a sense of detachment but not so much that it’s all strange again.