Building a trading bot?

[HavocXphere]

Looking for a new project...

Has anyone here built a trading bot before?

Current plan (hatched 5 mins ago - so still a bit under-cooked ):

Identify a single share that has good liquidity, get a cheap Azure instance to do twitter sentiment analysis and move cash in & out of the stock based on that. Likely very slowly - in fact the actual execution will probably be manual so the bot is more a trading signal.

Costs - 1 USD per buy / sell transaction on IB so if I move 2k each time then its 0.1% fee hit - as long I don't trade to fast the fees won't kill the whole plan. IB fee structure has essentially 10 trades "free"/sunk cost so realistically its even lower than 0.1%. Azure costs can be capped. Being stocks the stuff won't move wildly like FX etc so risk is tolerable if I screw something up. Don't think prospects of making money are particularly good, but seems reasonably safe so maybe break even & learn something?

Couple issues that I can think of thus far:
1) Backtesting twitter sentiment analysis is going to be +- impossible. That with the slow trading speed is a BIG problem for tweaking this.
2) Not particularly confident that existing sentiment libraries are particularly well suited to this given the cryptic language. Custom rolling something will be painful
3) Azure might not be efficient...considering running it from home
4) Timeframe is a big unknown. i.e. If twitter is positive right now does that mean staying in the stock for 3 hours, 3 days, 3 weeks? :erm: Difficult in light of backtesting issue above. Leaning towards 1 in/out decision a day

Ideas? Thoughts?

 

[Beachless]

I played around with a few scripts using a various input types on a fx demo site a few years ago. In the end I decided day trading just wasnt for me the scripts themselves had some succsess though. I've mostly settled on the long term strategy now and just go with the flow and invest as regularly as I can. Its a fun thing to do though if you have the time and you will learn an increadable amount by trying it.

 

[Idiosyncratic]

Wow, what a fantastic idea! I'm a little uncertain as to whether using Twitter as a base for determining sentiment would work in all cases... When was the last time you heard of Kumba (JSE:KIO) on Twitter? Correct me if I'm wrong. Personally, if I had the skill, I'd have a bot monitor those sites where the different firms analyze and publish "strong buy, buy, hold, sell and strong sell" ratings. Whenever a stock is given strong buy consensus - boom. If you're gonna limit things to the JSE things might be tricky - not as much material out there

 

[Thor]

I think bantu or bchip have built some trading robots not entirely the same as yours but still similar ie trigger sell or buy based on predetermined market speculation channels

 

[cguy]

Done that .

Sorry for being a downer, but my take in general is that unless someone is willing to buy high quality data (live and historical), fork out for the super computer hours, and become really good at data-science/ML, the odds of making money consistently are almost certainly worse than roulette.

Like you say, if you just want to learn, it's a fun project. You are likely getting really good at it if you get to the point that others are consistently beating you "to it" (vs say, the market randomly moving away from your predictions as easily as it moves toward them).

 

[HavocXphere]

Done that .

Was expecting that.

Sorry for being a downer, but my take in general is that unless someone is willing to buy high quality data (live and historical), fork out for the super computer hours, and become really good at data-science/ML, the odds of making money consistently are almost certainly worse than roulette.

Like you say, if you just want to learn, it's a fun project. You are likely getting really good at it if you get to the point that others are consistently beating you "to it" (vs say, the market randomly moving away from your predictions as easily as it moves toward them).

I know I have zero chance competing against the actual algo traders but I was under the impression that they're more in the short term space (i.e. seconds to minutes). Wouldn't longer holds (lets say a day) be +- devoid of algo traders and more full of day traders of the old school kind? Probably not even day traders...they're scared of overnight positions afaik.

Wow, what a fantastic idea! I'm a little uncertain as to whether using Twitter as a base for determining sentiment would work in all cases... When was the last time you heard of Kumba (JSE:KIO) on Twitter?

This would be US stocks - the fees are lower and much more activity. Stuff like Tesla gets 100s of mentions a day on twitter.

I think bantu or bchip have built some trading robots not entirely the same as yours but still similar ie trigger sell or buy based on predetermined market speculation channels

That almost appeals more to me...but that is solid in ML space and as per cguy that's full of big boys trying to take my lunch money. I know there are websites for this as well - i.e. you write code to buy/sell on a signal directly on the website. Cool idea but not sure I want to do that...

 

[cguy]

I know I have zero chance competing against the actual algo traders but I was under the impression that they're more in the short term space (i.e. seconds to minutes). Wouldn't longer holds (lets say a day) be +- devoid of algo traders and more full of day traders of the old school kind? Probably not even day traders...they're scared of overnight positions afaik.

Algo traders are in all the spaces, short, medium and long term holding. To make money consistently, you literally have to know something that they don't. Really, this means digging into the industry of the stocks you are trading and trying to build yourself a more educated world view than you competitors. Tesla - what do you think of their upcoming consumer part? Will it take off? How will it compete against the cheap electric cars produced by traditional car manufacturers? Is their battery technology really novel? Is there a patent block preventing competition from competing with the the charge time, battery life, battery weight, etc.? How much value does the brand name have in the consumer space? What are their margins? What predictions can you make about their margins? etc.

All the electronic information that doesn't require a huge amount of human research, the big funds already have - they've quantified it, and it's already factored into the price of every stock. The few day traders that make money have either by trial and error found some little niche that has been overlooked, or have actually done the research, and have a better idea of how to interpret news than anything that can be determined by the quantitative analysis of past market responses.

 

[HavocXphere]

Algo traders are in all the spaces, short, medium and long term holding. To make money consistently, you literally have to know something that they don't. Really, this means digging into the industry of the stocks you are trading and trying to build yourself a more educated world view than you competitors. Tesla - what do you think of their upcoming consumer part? Will it take off? How will it compete against the cheap electric cars produced by traditional car manufacturers? Is their battery technology really novel? Is there a patent block preventing competition from competing with the the charge time, battery life, battery weight, etc.? How much value does the brand name have in the consumer space? What are their margins? What predictions can you make about their margins? etc.

All the electronic information that doesn't require a huge amount of human research, the big funds already have - they've quantified it, and it's already factored into the price of every stock. The few day traders that make money have either by trial and error found some little niche that has been overlooked, or have actually done the research, and have a better idea of how to interpret news than anything that can be determined by the quantitative analysis of past market responses.

Well that's depressing. :/

Thanks for explaining.

edit...can't rep you unfortunately

 

[bchip]

Hi

I build algos to trade on the market.

The big secret to it is NOT the signal (eg your twitter analysis) its when you get in and when you get out.

The problem with your Twitter trading (nothing wrong with it) except
that you wont have good backtesting results. Backtesting is key to any system.

You need to be able to test where was a good point to enter (eg 2 days after the twitter feed or 20 min into the feed)
You need to then be able to test the position size and then finally most importantly -- where do you close?
Only when the sentiment changes to the other side or half way through?

The problem with all this testing is that you need reliable data, considering what little amount of "clean data" is available
I would therefore recommend rather focusing on understanding systems on a basic level before getting into something that complicated.

For example understand the Larry Connors RSI2 system...extremely simple
and only uses price, RSI and 1 moving average.

To make money consistently, you literally have to know something that they don't. Really, this means digging into the industry of the stocks you are trading and trying to build yourself a more educated world view than you competitors. Tesla - what do you think of their upcoming consumer part? Will it take off? How will it compete against the cheap electric cars produced by traditional car manufacturers? Is their battery technology really novel? Is there a patent block preventing competition from competing with the the charge time, battery life, battery weight, etc.? How much value does the brand name have in the consumer space? What are their margins? What predictions can you make about their margins? etc.

This advise is wrong...this is advise for someone trading discretionary stocks based on news,
this has nothing to do with system traders.

_________________________________________________________


Building a community of traders - http://www.myshares.co.za
Cassandra System
Results - 66% return in 30 days.

 

[bchip]

you literally have to know something that they don't. Really, this means digging into the industry of the stocks you are trading and trying to build yourself a more educated world view than you competitors. Tesla - what do you think of their upcoming consumer part? Will it take off? How will it compete against the cheap electric cars produced by traditional car manufacturers? Is their battery technology really novel? Is there a patent block preventing competition from competing with the the charge time, battery life, battery weight, etc.? How much value does the brand name have in the consumer space? What are their margins? What predictions can you make about their margins? etc.

All the electronic information that doesn't require a huge amount of human research, the big funds already have - they've quantified it, and it's already factored into the price of every stock. The few day traders that make money have either by trial and error found some little niche that has been overlooked, or have actually done the research, and have a better idea of how to interpret news than anything that can be determined by the quantitative analysis of past market responses.

This is what brokers, bankers and the media keep touting, but is incredibly inaccurate.
The average retailer cannot quantify strengths and weaknesses, or any other information accurately.
They also cannot quantify risk and decisions the major shareholders will make before it happens.
After you have done ALL those things you also need to figure out what the psycology/sentiment
is and which news matters are weighted higher than others.

For example right now I have 5 very convincing arguments why you need to be
bullish on the EURUSD, and at the same time I have 3 very convincing arguments
on what you have to be bearish.
Add sentiment, psychology, quantifying each arguement...its hours and hours
of work and chances you missing something or being inaccurate on one of them
is still high.

The fundamental methodology used to work in the 1950s - 1980s, Buffet got big on it.
It stopped working in the 80s onwards when the US was taken off the gold standard
and anyone could print any amount of money.
To truly understand this conversation though you would need to understand the difference
between causation and correlation...from there it becomes another 100 hour conversation

Just my 2c

 

[HavocXphere]

Hi

I build algos to trade on the market.

The big secret to it is NOT the signal (eg your twitter analysis) its when you get in and when you get out.

The problem with your Twitter trading (nothing wrong with it) except
that you wont have good backtesting results. Backtesting is key to any system.

You need to be able to test where was a good point to enter (eg 2 days after the twitter feed or 20 min into the feed)
You need to then be able to test the position size and then finally most importantly -- where do you close?
Only when the sentiment changes to the other side or half way through?

The problem with all this testing is that you need reliable data, considering what little amount of "clean data" is available
I would therefore recommend rather focusing on understanding systems on a basic level before getting into something that complicated.

For example understand the Larry Connors RSI2 system...extremely simple
and only uses price, RSI and 1 moving average.

Thanks for the advice!

hmm...seems everyone is in agreement - the twitter bot isn't going to work. I'll have a look at the Larry Connors you mentioned - maybe implement it pen & paper style for normal share buying. Bit wary of using simple RSI etc signals though...I'd imagine the big guys crunched the hell out of all possible permutations already.

This advise is wrong...

Pretty sure cguy works for a quant shop so wouldn't dismiss it too easily.

 

[cguy]

Hi

I build algos to trade on the market.

The big secret to it is NOT the signal (eg your twitter analysis) its when you get in and when you get out.

They're one of the same. You may be confusing "signals" with just external signals (e.g., twitter feed, other symbol feeds, other exchange feeds), vs internal signals (drawn from say the activity of the stock you're trading), but when to get in and when to get out is precisely driven by a combination of such signals (or at least it should be).

This advise is wrong...this is advise for someone trading discretionary stocks based on news,
this has nothing to do with system traders.

I was talking about how someone in HavocXphere's position could most likely make money, not how system traders (limiting their inputs) could make the most money. FYI, news can also be used to inform system traders, allowing them to be more effective - the trick is how to incorporate the information effectively.

 

[cguy]

This is what brokers, bankers and the media keep touting, but is incredibly inaccurate.
The average retailer cannot quantify strengths and weaknesses, or any other information accurately.

No, but someone else can. All it takes is one.

They also cannot quantify risk and decisions the major shareholders will make before it happens.

Once again, someone else can.

After you have done ALL those things you also need to figure out what the psycology/sentiment
is and which news matters are weighted higher than others.

For example right now I have 5 very convincing arguments why you need to be
bullish on the EURUSD, and at the same time I have 3 very convincing arguments
on what you have to be bearish.
Add sentiment, psychology, quantifying each arguement...its hours and hours
of work and chances you missing something or being inaccurate on one of them
is still high.

The fundamental methodology used to work in the 1950s - 1980s, Buffet got big on it.
It stopped working in the 80s onwards when the US was taken off the gold standard
and anyone could print any amount of money.
To truly understand this conversation though you would need to understand the difference
between causation and correlation...from there it becomes another 100 hour conversation

Just my 2c

To be clear. I don't believe that humans today can look at fundamental stats of a business (P/E ration, market cap, earnings history, etc.), and come up with anything not already factored in. For someone wanting to make money by themselves I lean more towards "Couldn't anyone see that Creative Labs would be dead, once hardware that could perfectly reproduce audio (to humans), was built?" - basically, a full understanding the sector and the technology for insight. There are obviously large external factors, but deep insight still provides an excellent signal if you can figure it out.

I'm also sure that both HavocXphere and myself know the difference between causation and correlation.

 

[FrankieK]

No, but someone else can. All it takes is one.



Once again, someone else can.



To be clear. I don't believe that humans today can look at fundamental stats of a business (P/E ration, market cap, earnings history, etc.), and come up with anything not already factored in. For someone wanting to make money by themselves I lean more towards "Couldn't anyone see that Creative Labs would be dead, once hardware that could perfectly reproduce audio (to humans), was built?" - basically, a full understanding the sector and the technology for insight. There are obviously large external factors, but deep insight still provides an excellent signal if you can figure it out.

I'm also sure that both HavocXphere and myself know the difference between causation and correlation.

And therein lies the crux. In this age of HFT algos that can scan a 10K, interpret it and act on it in milliseconds, what chance does your average person have? A person can do as much DD as possible but once you run up against these machines your DD is worthless. A great example is Skyworks Solutions and Apple. SWKS is a great company with incredible management that supplies hardware to AAPL. A small percentage of their income is derived from AAPL but most of it is from Samsung and Xiaomi etc. But because of their ties to AAPL the share price move virtually in lock step with AAPL.

So how does one invest in a great company when fundamentals no longer really count for anything, as bchip alluded to in his post about Buffet?

Edit to add: Just look at Nvidia the last few weeks. Announces agreement with Goldman and share price goes parabolic. Then Citron comes out with a bearish tweet and the whole thing comes apart at the seams.

 

[cguy]

And therein lies the crux. In this age of HFT algos that can scan a 10K, interpret it and act on it in milliseconds, what chance does your average person have? A person can do as much DD as possible but once you run up against these machines your DD is worthless. A great example is Skyworks Solutions and Apple. SWKS is a great company with incredible management that supplies hardware to AAPL. A small percentage of their income is derived from AAPL but most of it is from Samsung and Xiaomi etc. But because of their ties to AAPL the share price move virtually in lock step with AAPL.

So how does one invest in a great company when fundamentals no longer really count for anything, as bchip alluded to in his post about Buffet?

Algos, aren't going to have deep insights. Just because there are external factors, it doesn't mean that deep insight doesn't create a bias. All one needs is this bias to have an expectation of profitability. Pure systematic trading will run into algos though, and will likely lose money.

The reasons that Skyworks moves so tightly with Apple is certainly not some sort of random artifact of the times. More than likely there is some other driver of this relationship apart from how they earn their money, such as the probability that Apple could massively increase their revenue or that Apple could acquire them.

 

[Thor]

Algo traders are in all the spaces, short, medium and long term holding. To make money consistently, you literally have to know something that they don't. Really, this means digging into the industry of the stocks you are trading and trying to build yourself a more educated world view than you competitors. Tesla - what do you think of their upcoming consumer part? Will it take off? How will it compete against the cheap electric cars produced by traditional car manufacturers? Is their battery technology really novel? Is there a patent block preventing competition from competing with the the charge time, battery life, battery weight, etc.? How much value does the brand name have in the consumer space? What are their margins? What predictions can you make about their margins? etc.

All the electronic information that doesn't require a huge amount of human research, the big funds already have - they've quantified it, and it's already factored into the price of every stock. The few day traders that make money have either by trial and error found some little niche that has been overlooked, or have actually done the research, and have a better idea of how to interpret news than anything that can be determined by the quantitative analysis of past market responses.

Or they trade they the chart.

 

[cguy]

Or they trade they the chart.

Pretty difficult to make any money there. I would say that most that claim to be profitable were lucky enough to ride out a long term trend for a while. There's a lot of survivorship bias in the industry.

 

[Thor]

Pretty difficult to make any money there. I would say that most that claim to be profitable were lucky enough to ride out a long term trend for a while. There's a lot of survivorship bias in the industry.

Oh for sure, was just pointing out that some are more technical traders (charts) and others are speculative traders (gambling/luck).

 

[cguy]

Oh for sure, was just pointing out that some are more technical traders (charts) and others are speculative traders (gambling/luck).

For sure. My point is that a lot of those who think they are the former are actually the latter, they just don't know it.

 

[Thor]

So true