Hacker News new | comments | show | ask | jobs | submit login
Python For Finance: Algorithmic Trading (medium.com)
218 points by pastefka 11 hours ago | hide | past | web | 98 comments | favorite





The main issue I found in algo and financial aspects of programming is that the market is a zero sum game, and my intro knowledge of finance and algorithms, even when I know python, are no match for MIT PHD Quants who does it full time. There's no real way to compete with that, and therefore I would lose money, even if the data showed it might be successful in the future, firms and full time workers on algo trading would simply be faster, more focused, have more funding, and be able to quickly and constantly adapt at the scale an individual could not.

So despite the fact that the subject is interesting, I'd consider it a waste of time to try and gain anything but a basic understanding of the industry and how algo trading works.


An economist and a normal person are walking down the street together. The normal person says “Hey, look, there’s a $20 bill on the sidewalk!” The economist replies by saying “That’s impossible- if it were really a $20 bill, it would have been picked up by now.”

This quiq seems to support the original posters thesis, though, no? No one's going to make a living wandering the streets in search of $20 bills.

You are oversimplifying and overcomplicating at the same time. If it was an easy/obvious zero-sum game, we'd not have people going into finance at all.

MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".


Sorry, but this is like saying those CalTech PHD in Computers Science are the best, so no point for me to go into that field... This is like everything in life, not easy, but if you work on it it'll pay out in the end

I agree with most of what you're saying in principle, but it is very possible to identify alpha or acquire an edge that institutional market participants don't have. If you have access to data that most of the market does not, you can effectively trade on it. You can also effectively trade on a novel insight on a combination of data sources.

There is a lot of information asymmetry, and an individual is capable of capturing that without requiring a PhD or the resources of a large firm. That's not to say it's easy per se, but it's not hopeless. It requires special expertise or an unconventional approach.

Otherwise I agree that most people probably shouldn't attempt it (for risk tolerance reasons).


Dsacco, I really like your way of thinking, too bad we cannot agree on the cheap data sources :-) My hope is that one day you can see the world trough my glasses :-)

Even if it's a zero sum game (which it's not), most participants are not quant/algo based. As long as you can beat 50% percentile, you can make a profit.

Sure you may not be as profitable as top quant companies, but do you really mind that much?


Why on earth do you think the top quant companies will leave alpha on the table for you to snatch up?

That sounds too much like wishful thinking to me. Furthermore, if someone is really smart enough to beat the market consistently why on earth would they trade just on their own personal account? Work for a hedge fund and use other people's money to leverage your bets.


> Furthermore, if someone is really smart enough to beat the market consistently why on earth would they trade just on their own personal account?

Alternatively, if you have a strategy that empirically works, and a strategy for identifying such strategies, why take on investors to share the risk? You can scale up your own investment and leverage yourself with less regulatory oversight.

Why on earth do you think the top quant companies will leave alpha on the table for you to snatch up?

It's not so much that they leave alpha on the table, rather that there is so much alpha available, and the capacity constraints and reward profiles are so different for many of them, that individuals can prosper outside of a firm. They just normally don't, because they lack the same training (and because it's very competitive).


I'm curious to know who the loser was with the cryptocurrency rally this year?

You've forgotten you can introduce new units of whatever is being traded and quite often is & there are additional complications such as dividends, stock options, etc which makes your oversimplification lacking substance.


You can get an advantage if you focus on smaller markets or industries where it's not worth the time for the big quants to play in.

If the market is a zero sum game, then for every winning trade, there must be a losing trade. Obviously some teams must be winning consistently, which implies that some are losing consistently. This says that the consistent losers go out of business. So who are the winners going to trade with? By contradiction, it is not zero sum.

> Obviously some teams must be winning consistently, which implies that some are losing consistently.

That does not necessarily follow; markets are not static and players are free to enter and leave.


People like to say the market is a zero sum game, but I have always been suspicious of this truism.

Its fair to say that participants can leave the market, but in practice that isn't what we see. In practice there are firms doing this trading, and they are staying in business, and obviously making money.

Are they fleecing the little guy then? This explanation falls flat for me, i.e. for the amount of money they seem to be making, it would take a lot of small time participants losing everything every day. Most people I know aren't even active traders.

So why is it an accepted truism that the market is zero sum?


The missing puzzle piece here are dividends. Sure some people buy high and sell low and thus take a loss. But the market as whole can increase in value due to the external growth and profit.

>Are they fleecing the little guy then?

Yes, more or less (although that's just one way they make money, and probably not the most lucrative) And fortunately for them there are plenty of "little guys" ready to enter the market on a regular basis.


My point exactly, not the most lucrative. So what remains is the implicit assertion they are fleecing the big time guys. And somehow the big time guys remain in business... so what's going on?

If you want to build a real algorithmic trading strategy.. please pick up a real textbook such as Qian's Quantitative Equity Portfolio Management.

You should understand the following concepts at a minimum:

- Markowitz portfolio optimization (mean-variance analysis)

- Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors, etc)

- Alpha decay

- Time series analysis (autocorrelation, GARCH, ARMA processes)

- Basic price-based signals (momentum, volatility, value, etc) - etc, etc.


Personally, I struggle to see the competitive advantage Quantopian brings. They use retail brokerage platforms to facilitate trading, which rules out anything close to HFT. Then, they are tied to any financial data vendor (Morningstar in this case) to not offer too much visibility on the underlying data. As others have mentioned, this makes it tough to validate aspects like adjusted vs. as reported earnings, how delistings are handled, etc. From my experience, getting/making sure the data is accurate is a ton of work, even if it is from good sources and you can see all the actual data. The moment an investor/trader on the platform gets traction is the moment they want Compustat data, exchange data, Bloomberg for fixed income, and will trade through Instinet/Flextrade, etc. The moment the platform is successful is the moment Morningstar could pull the rug out from them. If someone has better understanding or knowledge on Quantopian in particular, I'd be interested to hear why.

My understanding is that Quantopian has no interest in being a platform for HFT. It's there to democratize trading strategies and test them in a sound way. They do a lot to have a good backtesting platform and clean the data that is available in the platform. If I wanted to automate a strategy, I'd have to figure out the Robinhood API and basically recreate what they have. Instead, I can use their platform, their backtests, their free data, and build strategies that efficiently lose money.

I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.


It sounds like day trading for script kiddies. There's no way to develop or execute a sound strategy with that level of access.

It's like a slot machine tournament. No one has an edge but some percentage of people will do well enough over a finite period to convince them to put real money behind their lucky algorithm.


I understand they don't, but the execution options could matter. Retail brokerage platforms don't actually send your orders to the exchange: http://www.schwab.com/public/schwab/nn/legal_compliance/impo.... If your turnover is very low, and your trade size is small, this shouldn't make an impact in liquid products, but only having market and limit orders can be very limiting. Granted, IB is probably the best retail platform, but it's not perfect. Even a smart beta strategy would likely have 4X turnover to an index. I do hope they succeed, and perhaps they begin offering a more open, premium offering to compete with Capital IQ, etc. I just have a hard time seeing how operating as a fund-of-funds essentially is a good business model considering the CAPEX they have to build the software. I could see it as a way to build, test, and then offer a premium back-testing platform or portfolio analysis/attribution tool once they've ironed out the bugs. There's a lot more money in that business than in fund-of-funds.

The main problem with Quantopian is that the data missing especially for delisted stocks, here are some examples you can test yourself: AA, CWH, NIHD, PANL, HTZ, DSL - I have a whole list I found very fast, just imagine what other mistakes could be there... And the other problem is that they are my competitor, so I would never give them my trading strategy or ideas I am testing at the moment...

This is false. Quantopian sources their historical and real time data from Nanex, and the data is explicitly free of survivorship bias (i.e. they maintain data for delisted equities).

I don't know what you verified, but it directly contradicts my own experience and the FAQ: https://www.quantopian.com/faq#data-sources

To your second point, as another commenter said Quantopian doesn't see your algorithm unless you give them permission, as outlined in the terms of service (which are very readable, by the way).

NB: I am not unaffiliated with Quantopian and don't use the platform for my own trading, but I have tried it out before.


Please go back and check and you'll se the data is missing for the symbols stated above, and I am sur for many more based on my limited test... everyone can see I am right by trying to request the data, nothing false about that...

> everyone can see I am right by trying to request the data, nothing false about that...

Oh, they can? Well that's funny, because I just went onto Quantopian and checked for myself.

http://imgur.com/a/VwMUJ

http://imgur.com/a/VV68C

Gee, would you look at that...they are there.

This took me all of five minutes. Do you have anything else I can easily disprove while I'm at it?


The problem is the data is missing for specific dates... I just can imagine what else is missing or it is wrong...

This is really trying my patience. You made a clearly incorrect claim without mentioning specific dates and challenged me to disprove it. When I did disprove the claim, you shifted the goalposts with this backpedaling remark about how only specific dates are missing. You continue to make these claims and put the burden of proof on everyone else because you can't be bothered to defend them when you're challenged.

I'm not searching for a needle in a haystack for you. Make a falsifiable claim, and make the entire claim, with the specific dates you're talking about, or stop this astroturfing against Quantopian and QuantConnect that you're doing.


My goal is to show people how start to think for themselves and hopefully create lifestyle business around trading or invest for themselves instead of using "buy and hope strategy" and how to do it without losing a single penny while learning and to show them the barrier for entry is pretty low... But you are right, why bother when every time I gave an idea, the day traders with the thousands of dollars data feeds are all over my posts... no good deed goes unpunished here at HN I guess :-(

Shitting on other people products with vague and inaccurate criticisms isn't a good deed, it's just bad form. People don't need you to "show them how to think for themselves." The other guy is right, either make a sound and specific criticism or don't say anything. You did exactly what he said, you made a bad argument which he refuted, then you moved the goal post so as not to appear wrong, while still leaving out the necessary detail to disprove your new vague claim, rather than admitting your mistake and making a better argument that included specifics about what is missing.

What dates do you see missing data for?

The trading strategy stuff has been brought up before. They say when evaluating algorithms they only look at the alpha, beta, etc. and not the algorithm itself. While the system can see what you're running, I think they work pretty hard to let your code be yours. I'm definitely not very advanced in algorithmic trading, so I see how there could be some data issues that I wouldn't come across.

They're searching for "alpha" strategies. Then they buy/share profit with the creator... I have a hunch that they are feeding these good strategies to some type of ML in order to become more efficient and profitable.

QuantConnect recently announced full python library support; and we have launched https://www.quantconnect.com/tutorials to help people write quantiative strategies in Python.

QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started.

The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.


Hey Jared, two questions:

1. Any chance of a Robinhood integration à la Quantopian?

2. How are the architectural revisions[0] coming along?

Also some totally unsolicited feedback:

If I'm being completely honest, I found it difficult to get going with QC. The documentation is decent, but there's not enough to avoid having to review LEAN source right off the bat. The examples also tend to mix helper classes with lower-level functionality, and that can create confusion.

The framework itself feels a bit over-reliant on OOP. Some aspects feel too tightly coupled, others too little. Obviously LEAN has been around for many years now, so architectural baggage is perfectly understandable.

A total rewrite I'm sure isn't feasible, though I'd suggest the following ethos in any case:

a) Design primitive user-accessible data structures with virtually no inbuilt functionality.

b) Build low-level components that operate using those data structures.

c) Build high-level components that compose low-level components.

d) Allow users to author their own components, and to compose components of any type however they see fit.

Pretty sure you're already on that track in a sense, so it's good to see things headed in the right direction. What keeps me from writing a custom framework is the data, the fact QC does a ton of grunt work, and that it's well-tested.

tl;dr Please break apart the monolithic QCAlgorithm class as much as you can! :)

[0] https://www.quantconnect.com/forum/discussion/1816/qc-algori...


When promoting your company, please say so, also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...

Sorry SirL! I edited it within 10 sec to be explicit but you must've refreshed before I'd updated it =)

We provide FX/CFD data for free download; the other data is restricted by the exchanges sadly so we can't make it available. Instead we put the tools we used to make it into LEAN format into LEAN (/Toolbox) so you can purchase it and convert it yourself.


Thanks jaredbroad, just out of curiosity, what are the average yearly returns on your top 3 users, how much are you investing with them and how do you split the profits?

>When promoting your company, please say so ...

His bio and pronoun usage said as much.

>also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...

Can you not load up the data and test it however you wish? They even say where they get their data from.


No you cannot

First of all, you're criticizing QuantConnect's data when you claim to get your data from ebay.

Second, yes you absolutely can verify QuantConnect's data. You can use it as much as you want within the context of their platform, you just can't download the data en masse from their platform and use it on your own. But if you have tick data (equities) or minute data (options) yourself, you can certainly verify it (which of course, you don't, because you think ebay is a good source for financial data).

I am going to continue griefing you in threads like this where you spread blatant misinformation, because at this point I'm convinced you have an ulterior motive or are in fact selling this data you keep talking about on ebay.


Unfortunately you cannot display QC tic data to be able to very it against you broker for example. As I said before if you have a good and cheap source, please share it with everyone...

> good and cheap source

Pick one. You don't get both. I'm happy to share where you can get real data from.

https://www.tickdata.com

https://datashop.cboe.com

http://www.nanex.net/nxcore.html

https://quantquote.com

No affiliation with any of these.


On a side note, the market continues to do well and I've been noticing this trend of active-trading, real-estate investing gurus crawl out of the wood work selling services.

Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.


> Passive investing using index funds is definitely not sexy, but it gets the job done.

Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.


Here's a dataset on Kaggle that can be used in this process https://www.kaggle.com/biomimic/periodic-table-of-elements-m...

I went to the Quantopian conference for their basic training on algorithmic trading. This blog post was pretty much what they covered, intro to pandas and a simple strategy. There is a lot of educational material on their site too (which is what you ended up getting in the paid training).

My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python.

1. https://lectures.quantecon.org/py/


The problem with learning finance that it takes a lot og effort. A good place to start would be taking the 3 CFA exams. After about 1000 hours of study, you will have a basic grasp of finance.

This is utterly the wrong way to learn finance for quantitative trading. That would be an unproductive use of time.

Read Options, Futures and Other Derivatives and Algorithmic Trading and DMA and you basically know everything you'll get from public sources that could be meaningful for trading.

If he knows how to code and is looking for "mathematics of finance" he should start with those, not the CFA.


That is your opinion, but if you look at the resumes of scientific active equity fund managers and quants, they all have it. Just understanding the quantitative side leads to vast underperformance over the course of market cycles...

Quants don't usually have CFAs. A phD in physics is more common to see. CFAs help you working in a real financial environment where you need to be aware of a lot of stuff. If you want to day trade at home, it's close to useless. Please nobody gets a CFA if what you want to do is learning day trading.

What resumes? Can you show me these vast numbers of resumes with CFAs? Are you talking about fund managers or quants? If your goal is develop quantitative trading strategies, you shouldn't be getting a CFA because it's superfluous.

http://news.efinancialcareers.com/us-en/150266/why-the-cfa-w...

On another note, even if what you were saying were true (and it isn't), it wouldn't demonstrate that it's useful to have the CFA. Certifications are usually helpful for HR or for regulation, not for actual knowledge. And everything you need to know about finance can be learned in far less than 1000 hours for trading if you're in a research or development role.


Linkedin. Go there, search for scientific active equity (that is what a professional quant manager is called).

Sort by who manages real money and who doesnt. Anyone managing $1bn or more all has it. That is not very much money to manage.


I feel like we're talking past each other. I'm talking about a role where you develop strategies as either a researcher or a trader. Higher up the chain, sure, I can admit there's a utility to the CFA.

I think the main issue is that you need a solid grasp of the qualitative side of finance to build working quant models. Pms are responsible for quant model construction and they all get cfas because that is base level finance understanding.

Did you find the conference worth it ? I was thinking of going to it as well or even going to the online version since it's a lot cheaper but ended up not taking part.

those are indeed good lectures but they are not math for finance per se - more like dynamic programming for econ/finance. my impression is that finance as such is a hodge-podge of econ/accounting/business.

This is nothing more than gambling. Let's say you have 20k to play with. You would be far better off in the long run to put 15k in Wealthfront and use the other 5k as a bankroll of 25 buyins for 1/2 no limit holdem to learn the game.

I have a simple manual algorithm that took me from 650€ to 1450€ now (as test) in 2 months. Does someone here have experience with algo trading in c#?

I won't say I have "experience" but quantconnect.com is as good a platform as I've found for C#. Their engine is open source so you don't have to rely on them either, though you'll need to provide your own data if you run it yourself.

QuantConnect also offers tick-level data, though not full resolution if memory serves. Still way ahead of Quantopian in any case.

Futures have millisecond timestamps (trades/quotes), equity trade ticks are rounded to the nearest second, cfd/forex are millisecond quote bars. For options we have minute resolution data =)

Worth noting though that Options data is super hard to get running on QuantConnect due to the memory limits and lack of index options.

Agree, we're working on making it easier now. We patched a bug today which should make it more efficient.

please use a disclaimer if and when you are promoting your own company...

So, I got semi-seriously interested in this around the end of last year.

My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios.

I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations?

How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat?

Me:

1) Build some simple forecasting NNs.

2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure.

3) Get impatient -- get some "bright" ideas, do some manual trading.

4) Make some money on the first day, lose double that on the next two days.

5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading.

6) Back to coding infrastructure. Get bored ... do some manual trades.

7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies.

8) Enjoy my 25% returns.


How is algorithmic trading not equivalent to astrology?

Nothing can be predicted because there's way too many confounding factors.


I have a mean reversion strategy based on the comparing the results of several types of sentiment analysis in real time. It earns between 3 - 15% in options trading every few days with a win rate of around 70% and an average holding time of a few hours. Stop losses are at -70% or so.

Algorithmic trading is very difficult, but it is empirical. Information asymmetry exists in the market, and if you can capture it you profit. I don't know how capacity constrained my strategy is (I think it would be difficult to work this strategy profitably with over $10M depending on how much effort you put into execution; my execution is unsophisticated and I'm working with two orders of magnitude less than that), but it's working.


> Stop losses are at -70% or so.

Am I reading that right, you lose one trade you lose 70% of your capital? Sounds like a very very dangerous strategy with a near 100% certainty of blowing up the account. The optimal trade size for equity growth (which is itself hugely risky) is vastly smaller than that.


Hah, no that's a typo, sorry my mind was a bit scattered in this thread. I meant to type that stop losses are at -15% or so (of the position, not the portfolio).

There is a separate risk management metric in play that automatically closes positions and pauses the strategy after a specific number of losses in a row, or after the strategy's capital allocation drops below a certain percentage (whichever comes first).

The other metrics are as stated.


> -15% or so (of the position, not the portfolio).

Oh... HUGE difference. What percentage of the portfolio is the position? How long have you run this strategy; I ask because those are huge returns that indicate to me you're taking huge risk which can work short term but nearly always wipes you out over a sufficient amount of time.

> It earns between 3 - 15% in options trading every few days with a win rate of around 70% and an average holding time of a few hours.

If those returns could be held for any length of time you'd be wealthy in no time.


> Oh... HUGE difference. What percentage of the portfolio is the position? How long have you run this strategy; I ask because those are huge returns that indicate to me you're taking huge risk which can work short term but nearly always wipes you out over a sufficient amount of time.

No single position can be more than 10% of the portfolio, there is no limit on concurrent positions, and the strategy cannot use more than 20% of the account's capital. Each position has a profit target, normally a 5% increase on the cost basis, and when it's opened a GTC order is immediately submitted to flip it. But this isn't a market-making algorithm or something that requires serious latency considerations - the target holding period for each position is less than five days, after which if the position is still open it's flagged. The strategy has been running continuously for only eight months; prior to that it was backtested with historical data before launching live. It used to be that the mean holding time was around 2 days, but lately it's been flipping positions in fairly short intraday scales (an hour or less), which means I'll be adding more risk accountability to it.

> If those returns could be held for any length of time you'd be wealthy in no time.

Yes...however this strategy has generated less than $30,000 in profits since its launch (on an initial outlay of $10,000), and I don't reinvest profits.

This is an options trading algorithm, and I don't think there is enough liquidity available in the targets to significantly ramp up capital from here without adding on more leverage (and risk). The other difficulty is that a confluence of factors needs to happen somewhat simultaneously in order for a candidate equity to become a target.

Due to this, it's more accurate to say that the strategy generates a few hundred dollars weekly to biweekly (on average). I may be overcautious, but this is primarily a research project for me. I would not actually be trading with real capital were it not for the fact that I'd like to demonstrate the results in the future.


Algo trading typically utilises technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons, or fundamental analysis (e.g. algorithmically valuing and pricing options based on underlying fundamental data, sentiment analysis, etc.), and buying/shorting as appropriate. It is based on scientific methods: empirical evidence being used to validate hypothesis that produce results that can be shared and repeated by others.

Astrology is story-telling with no scientific grounding.


> technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons

Technical analysis is merely another name for hindsight bias. Those patterns only look like they repeat in hindsight because you're ignoring all the failed patterns that setup right but failed to play out and thus don't look like the pattern in hindsight.

Technical analysis is exactly like astrology and is in no way grounded in science. Quantitative analysis is based in the scientific process, technical analysis is superstition created by visual traders who think they see patterns but fail to understand biases. You are confusing the two, technical analysis is a pattern library of nonsense like "head and shoulders" or "dead cat bounce" based purely on visual patterns in price data that only "appear" to repeat due to hindsight bias. It is in no way remotely scientific.

I've spend thousands of hours testing trading patterns in Forex, they're all nonsense that lose as much as they win over any decent sample size which equates to random.

Edit: to the guy below, You should ask for sources that back up the claims that it does work. You won't find any, it's trader superstition, not science. You don't have to prove things don't work; you have to prove they do. At least if you want to call it based in science. However, the field of quantitative analysis itself is proof technical analysis doesn't work; it's what happened with people who actually knew math got into trading and discovered the technical analysis superstition that visual traders who lack math skills made up.


Can you provide sources that back up your claim with studies of inefficiency of technical analysis?

I'm not busting your balls, I actually agree with you but I haven't seen it really proven.


So how do you explain this:

When filing for its IPO in March 2014, it was disclosed that during five years Virtu Financial made profit 1,277 out of 1,278 days, losing money just one day.


The net gain in stock trading is usually positive in the long run, even if you just buy and sell randomly. Just look at the DJIA.

It is a great article, but why on earth someone will use a service like Quantopian or similar service?

They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy?

Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc...

The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...


Because they do so much work upfront for you. I worked in algorithmic trading for a bit over 10 years but left in 2014. I spent a few months in my spare time building out a trading and backtesting platform, pulling data off of yahoo finance, connecting to Interactive Brokers via their C++ API, etc. It was all very slow and tedious, and during my googling one day, I come across quantopian. I felt like an idiot- they already had free access to morningstar with a decent API around it, tons of other datasets you could subscribe to, and a fully fleshed out backtesting platform with risk measures, pretty graphs, etc.

It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's.

Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets.

One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well.

If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.


My experience is unfortunately different, I found the data for some delisted stocks like HTZ missing which will massively screw the past results due to the survivorship bias...

Can you please prove this or stop saying it? Quantopian's data is from Nanex and is free of survivorship bias, as I mentioned to you in another comment.

Yes I can prove it, go and request historical data for symbols like AA, HTZ and you'll see yourself

Mostly a matter of trust. I trust Quantopian more than I trust some random asshole on eBay purporting to sell accurate historical data. Plus, Quantopian is convenient.

This is the cheapest data I can find and you can do random checks to verify the data...

if you know a cheaper or even better a free place please advice...


Have you done this? If so it would be great to get your perspective

This commenter comes into every single thread about trading and talks about buying data from ebay, then consistently demonstrates that he doesn't know the first thing about due diligence on financial data. Each time I try to ask him about his data quality or methodology at even a high level, he responds by accusing me of wanting to steal his work or stop the democratization of data.

I'm going to reiterate this right now: you are not going to find data valuable enough for trading insights on ebay. Go look at the prices of historical data for various time resolutions at TickData or CBOE Livevol and then ask yourself if cheap, competitive data on ebay is too good to be true.

If the data is competitive, it's expensive. If it's not competitive, you shouldn't be paying for it, especially not from ebay. The first time I read about him doing this I was puzzled, now I'm getting increasingly frustrated because it's blatantly false information.


Unless you are day trading you don't need such a granular data format, so please stop saying you need to spend thousands and thousands of dollars to be able to back test a trading strategy

As far as I understand historical daily data is easy to get from yahoo, google, etc. Data is only difficult to find/expensive when you need tick data.

If your data doesn't need to operate on an intraday scale, you can get perfectly accurate real time or historical data for free just by using a reputable broker like Interactive Brokers. At that point there's no need to buy the data on ebay, so I still don't understand why you ever would.

Actually I have checked IB API and it is very restrictive, e.g. you cannot request let say all daily data for all symbols for 2007 for example or even for one day for that matter...

Yes, I am doing it as a side project and I am looking to retire in a few years and live off that...

I second the above comment; you should write about the process and how it is working out for you.

I'll third this comment. Some more details on what you trade, where you trade, how long you've been doing it, performance metrics, etc... would be fascinating.

As you can see I am paranoid, because the more people trading the same strategy, the less effective it becomes, not to mention, if someone knows your trading strategy it can trade against you... it’s a zero sum game after all...

All that said, here are the 2 most important advises I wish someone told me when I started years ago:

Have an iron clad risk management strategy for your portfolio, e.g. if you are willing to risk 1% of your capital on any given day and tomorrow happens to be the Black Monday and all stock go down 50% you still have to be sure you’ll loose only 1% if you have to sell everything... This is where the back data comes into play – the goal is to back test not how much money you are going to make, but how much money you are going to loose in the worst case scenario if you have to liquidate everything and move to all cash. If you have this working 100%, you just need a trading strategy with a minimal edge and you’ll make money in the long run in the market and the compounding effect will really make you a rich person (rich person to me is someone who makes more money then he spends, WITHOUT working more than 1 hour a day)

And the second advice is never ever trust and buy a trading strategy from someone, the fact he is selling or "teaching" a trading strategy means it is not working and he is trying to make some money by scamming people and selling the dream, all those trading educators cannot make money trading and that’s why they are coming with up with those fake trading courses... just check this website to see the magnitude of this scam and how people are loosing their life savings... I just don’t know why FBI, SEC and the government is not doing something to stop those guys...

https://www.tradingschools.org/

Good luck and as I said before, just treat is as a hobby at first and spend the time testing and you’ll find something that is working for you...

EDIT: some spelling errors fixed.


> more people trading the same strategy, the less effective it becomes

you should pick a more scalable strategy, the kind of strategy that becomes more profitable the more you talk your own book.


Not everyone can build their own backtesting tool. I played around with quantopian for a little while and did some live trading with quantopian + robinhood and I never would have been able to do that without quantopian

I am not a developer, but using my brokers API, examples and python, it took me few months to build a very robust and automatic trading system... I did learn python by back testing my trading ideas... To use Quantopian you still have to learn python, so I am not sure where is the benefit, considering all the problems mentioned here...



Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: