I never saw a need for this in HFT. In my experience, GPS was used instead, but there was never any critical need for microsecond accuracy in live systems. Sub-microsecond latency, yes, but when that mattered it was in order to do something as soon as possible rather than as close as possible to Wall Clock Time X.
Still useful for post-trade analysis; perhaps you can determine that a competitor now has a faster connection than you.
The regulatory requirement you linked (and other typical requirements from regulators) allows a tolerance of one second, so it doesn't call for this kind of technology.
> I never saw a need for this in HFT. In my experience, GPS was used instead, but there was never any critical need for microsecond accuracy in live systems.
> The required accuracy (Tables 1 and 2 in that document)
no, Tables 1 and 2 say divergence, not accuracy
accuracy is a mix of both granularity and divergence
regardless, your statement before:
> The regulatory requirement you linked (and other typical requirements from regulators) allows a tolerance of one second, so it doesn't call for this kind of technology.
> accuracy is a mix of both granularity and divergence
I respectfully disagree.
In context, "granularity" is nothing more than a resolution constraint on reported timestamps. Its inclusion adjacent to the specified "divergence from UTC" is a function of market manipulation surveillance objectives as discussed in preamble item (2), and really doesn't have anything to do with accuracy proper.
But the reason factoring numbers is used as the quantum benchmark is exactly that we have a quantum algorithm for that problem which is meant to scale better than any known algorithm on a classical computer.
So it seems like it takes an exponentially bigger device to factor 21 than 15, then 35 than 21, and so on, but if I understand right, at some point this levels out and it's only relatively speaking a little harder to factor say 10^30 than 10^29.
Why are we so confident this is true given all of the experience so far trying to scale up from factoring 15 to factoring 21?
> Why are we so confident this is true given all of the experience so far trying to scale up from factoring 15 to factoring 21?
I don't think we have any "real" experience scaling from 15 to 21. Or at least not in the way shor's algorithm would be implemented in practise on fault tolerant qubits.
We haven't even done 15 yet in a "real" way yet. I susect the amount of time to factor 15 on fault tolerant qubits will be a lot longer than the time to go from 15 to 21.
The algorithm in question is a hypothetical algorithm for a hypothetical computer with certain properties. The properties in question are assumed to be cheap.
In the case of quantum algorithms in BQP, though, one of those properties is SNR of analog calculations (which is assumed to be infinite). SNR, as a general principle, is known to scale really poorly.
> In the case of quantum algorithms in BQP, though, one of those properties is SNR of analog calculations (which is assumed to be infinite). SNR, as a general principle, is known to scale really poorly.
As far as i understand, that isn't an assumption.
The assumption is that the SNR of logical (error-corrected) qubits is near infinite, and that such logical qubits can be constructed from noisey physical qubits.
There are several properties that separate real quantum computers from the "BQP machine," including decoherence and SNR. Error-correction of qubits is mainly aimed at decoherence, but I'm not sure it really improves SNR of gates on logical qubits. SNR dictates how precisely you can manipulate the signal (these are a sort of weird kind of analog computer), and the QFTs involved in Shor's algorithm need some very precise rotations of qubits. Noise in the operation creates an error in that rotation angle. If your rotation is bad to begin with, I'm not sure the error correction actually helps.
> The assumption is that the SNR of logical (error-corrected) qubits is near infinite, and that such logical qubits can be constructed from noisey physical qubits.
This is an argument I've heard before and I don't really understand it[1]. I get that you can make a logical qubit out of physical qubits and build in error correction so the logical qubit has perfect SNR, but surely if (say the number of physical qubits you need to get the nth logical qubit is O(n^2) for example, then the SNR (of the whole system) isn't near infinite it's really bad.
[1] Which may well be because I don't understand quantum mechanics ...
The really important thing is that logical qbit error decreases exponentially with error correction amount. As such, for the ~1000 qbit regime needed for factoring, the amount of error correction ends up being essentially a constant factor (~1000x physical to logical). As long as you can build enough "decent" quality physical qbits and connect them, you can get near perfect logical qbits.
Having demonstrated error correction, some incremental improvements can now be made to make it more repeatable and with better characteristics.
The hard problem then remains how to connect those qubits at scale. Using a coaxial cable for each qubit is impractical; some form of multiplexing is needed. This, in turn, causes qubits to decohere while waiting for their control signal.
You're correct but the complexity is in the things you're ignoring for simplicity.
This game is constructed such that the questions you can ask are not arbitrary, so you cannot choose them to always produce one bit of entropy (you need to frame your questions as ten matchups in parallel, using all the contestants exactly once) and the number of bits you need may indeed not be an integer.
Because you can't choose your questions to partition the state space arbitrarily, that affects not just the question you ask today, but also previous days: you want to leave yourself with a partitionable space tomorrow no matter what answers you get today.
In the Guess Who analogy, it's against the rules or at least the spirit to ask "does your character have a name which is alphabetically before Grace?". That would allow a strategy which always divides the state space exactly in two.
> and the number of bits you need may indeed not be an integer.
That's true but not relevant; needing a non-integer number of bits makes it possible for some games to end one turn faster than other games, but it doesn't make it possible for a maximum-entropy strategy to have more variation than that one maybe-necessary-maybe-not turn. The scenario described by my parent comment still isn't possible.
> This game is constructed such that the questions you can ask are not arbitrary, so you cannot choose them to always produce one bit of entropy (you need to frame your questions as ten matchups in parallel, using all the contestants exactly once)
You're confusing two types of questions. The question where you're targeting one bit of information is "are Matt and Felicia a match?", just one pair.
A full proposed matchup of n pairs has n+1 possible answers and so your goal is to produce log₂ (n+1) bits. (Better conceptualized as one base-n+1 "bit".) I agree that it's not obvious how to do this or to what extent it's possible.
Because everyone would cheat and include a program that sprays cold water for a few seconds, using minimal energy but not actually getting your dishes clean.
This article both undersells and oversells the technical challenge exchanges solve.
First, it is of course possible to apply horizontal scaling through sharding. My order on Tesla doesn't affect your order on Apple, so it's possible to run each product on its own matching engine, its own set of gateways, etc. Most exchanges don't go this far: they might have one cluster for stocks starting A-E, etc. So they don't even exhaust the benefits available from horizontal scaling, partly because this would be expensive.
On the other hand, it's not just the sequencer that has to process all these events in strict order - which might make you think it's just a matter of returning a single increasing sequence number for every request. The matching engine which sits downstream of the sequencer also has to consume all the events and apply a much more complicated algorithm: the matching algorithm described in the article as "a pure function of the log".
Components outside of that can generally be scaled more easily: for example, a gateway cares only about activity on the orders it originally received.
The article is largely correct that separating the sequencer from the matching engine allows you to recover if the latter crashes. But this may only be a theoretical benefit. Replaying and reprocessing a day's worth of messages takes a substantial fraction of the day, because the system is already operating close to its capacity. And after it crashed, you still need to figure out which customers think they got their orders executed, and allow them to cancel outstanding orders.
Once sequencing is done, the matching algorithm can run with some parallelism.
For example, Order A and order B might interact with eachother... but they also might not. If we assume they do not, we can have them processed totally independently and in parallel, and then only if we later determine they should have interacted with each other then we throw away the results and reprocess.
It is very similar to the way speculative execution happens in CPU's. Assume something then throw away the results if your assumption was wrong.
Off the cuff, id expect this leads to less improvement than you might think. The vast majority of orders, especially orders arriving in sequence close to one another, are likely on a small set of extremely liquid symbols, and usually all for prices at or near the top of the book for those symbols.
Happy to discuss more, might be off the mark... these optimizations are always very interesting in their theoretical vs actual perf impact.
in high scale stateless app services this approach is typically used to lower tail latency. two identical service instances will be sent the same request and whichever one returns faster “wins” which protects you from a bad instance or even one which happens to be heavily loaded.
I'm not sure I follow. In this instance we're talking about multiple backend matching engines... Correct? By definition they must be kept in sync, or at least have total omnipotent knowledge about the state of all other backend book states.
Yes, I was going to note that this doesn't necessarily apply on derivatives exchanges. But
a) I don't know of any exchange where this could be true for specifically Apple and Tesla, so the example is OK
b) you can still get some level of isolation, even on commodities exchanges you can't typically affect the gold book with your crude oil order (the typical case is that your order to buy oil futures in 2027 matches against someone selling oil in 2026, plus someone selling a calendar spread)
c) for exchanges that do offer this kind of functionality, one of the ways they deal with high volumes is by temporarily disabling this feature.
Commercial products are run by product managers: they do whatever the business needs that day, and if it doesn't work for most inputs, "that's fine, our users will only ever need addition". Fun open source projects, run by the same programmer who does the implementation, obsess over finding the generic solution to inverting a function and end up with a version that isn't useful for anyone's specific case.
After what release? Type checkers already support the various TypedDict PEPs, even those that haven't been accepted yet. Yet, none of them infers TypedDict types.
Not sure we're talking about the same thing. Inline TypedDicts are already in the process of being formalized, see https://peps.python.org/pep-0764/ , and have experimental support in e.g. Pyright.
Ah, I didn't know about PEP 795. I don't really know if I like it though. It looks like something someone invented because they don't want to write classes or specifically want to be able to access a data structure like a dict, for whatever reason.
It basically provides data types, but also ensures that you have no easy way to reuse them, and it will miss cases where two data structures happen to have the same signature.
It's getting a little messy when we have class, namedtuple and TypedDict, which all can do much the same thing.
It looks like the wheels could stay attached to the outside of the suitcase. It means leaving it in the less standard configuration to pull around (landscape rather than portrait) but that can be overcome with a longer handle.
reply