Best of the Week
Most Popular
1. Investing in a Bubble Mania Stock Market Trending Towards Financial Crisis 2.0 CRASH! - 9th Sep 21
2.Tech Stocks Bubble Valuations 2000 vs 2021 - 25th Sep 21
3.Stock Market FOMO Going into Crash Season - 8th Oct 21
4.Stock Market FOMO Hits September Brick Wall - Evergrande China's Lehman's Moment - 22nd Sep 21
5.Crypto Bubble BURSTS! BTC, ETH, XRP CRASH! NiceHash Seizes Funds on Account Halting ALL Withdrawals! - 19th May 21
6.How to Protect Your Self From a Stock Market CRASH / Bear Market? - 14th Oct 21
7.AI Stocks Portfolio Buying and Selling Levels Going Into Market Correction - 11th Oct 21
8.Why Silver Price Could Crash by 20%! - 5th Oct 21
9.Powell: Inflation Might Not Be Transitory, After All - 3rd Oct 21
10.Global Stock Markets Topped 60 Days Before the US Stocks Peaked - 23rd Sep 21
Last 7 days
How to Get Rich in the MetaVerse - 20th Jan 21
Should you Buy Payment Disruptor Stocks in 2022? - 20th Jan 21
2022 the Year of Smart devices, Electric Vehicles, and AI Startups - 20th Jan 21
Oil Markets More Animated by Geopolitics, Supply, and Demand - 20th Jan 21
WARNING - AI STOCK MARKET CRASH / BEAR SWITCH TRIGGERED! - 19th Jan 22
Fake It Till You Make It: Will Silver’s Motto Work on Gold? - 19th Jan 22
Crude Oil Smashing Stocks - 19th Jan 22
US Stagflation: The Global Risk of 2022 - 19th Jan 22
Stock Market Trend Forecast Early 2022 - Tech Growth Value Stocks Rotation - 18th Jan 22
Stock Market Sentiment Speaks: Are We Setting Up For A 'Mini-Crash'? - 18th Jan 22
Mobile Sports Betting is on a rise: Here’s why - 18th Jan 22
Exponential AI Stocks Mega-trend - 17th Jan 22
THE NEXT BITCOIN - 17th Jan 22
Gold Price Predictions for 2022 - 17th Jan 22
How Do Debt Relief Services Work To Reduce The Amount You Owe? - 17th Jan 22
RIVIAN IPO Illustrates We are in the Mother of all Stock Market Bubbles - 16th Jan 22
All Market Eyes on Copper - 16th Jan 22
The US Dollar Had a Slip-Up, but Gold Turned a Blind Eye to It - 16th Jan 22
A Stock Market Top for the Ages - 16th Jan 22
FREETRADE - Stock Investing Platform, the Good, Bad and Ugly Review, Free Shares, Cancelled Orders - 15th Jan 22
WD 14tb My Book External Drive Unboxing, Testing and Benchmark Performance Amazon Buy Review - 15th Jan 22
Toyland Ferris Wheel Birthday Fun at Gulliver's Rother Valley UK Theme Park 2022 - 15th Jan 22
What You Should Know About a TailoredPay High Risk Merchant Account - 15th Jan 22
Best Metaverse Tech Stocks Investing for 2022 and Beyond - 14th Jan 22
Gold Price Lagging Inflation - 14th Jan 22
Get Your Startup Idea Up And Running With These 7 Tips - 14th Jan 22
What Happens When Your Flight Gets Cancelled in the UK? - 14th Jan 22
How to Profit from 2022’s Biggest Trend Reversal - 11th Jan 22
Stock Market Sentiment Speaks: Are We Ready To Drop To 4400SPX? - 11th Jan 22
What's the Role of an Affiliate Marketer? - 11th Jan 22
Essential Things To Know Before You Set Up A Limited Liability Company - 11th Jan 22
NVIDIA THE KING OF THE METAVERSE! - 10th Jan 22
Fiscal and Monetary Cliffs Have Arrived - 10th Jan 22
The Meteoric Rise of Investing in Trading Cards - 10th Jan 22
IBM The REAL Quantum Metaverse STOCK! - 9th Jan 22
WARNING Failing NVME2 M2 SSD Drives Can Prevent Systems From Booting - Corsair MP600 - 9th Jan 22
The Fed’s inflated cake and a ‘quant’ of history - 9th Jan 22
NVME M2 SSD FAILURE WARNING Signs - Corsair MP600 1tb Drive - 9th Jan 22
Meadowhall Sheffield Christmas Lights 2021 Shopping - Before the Switch on - 9th Jan 22
How Does Insurance Work In Europe? Find Out Here - 9th Jan 22
MATTERPORT (MTTR) - DIGITIZING THE REAL WORLD - METAVERSE INVESTING 2022 - 7th Jan 22
Effect of Deflation On The Gold Price - 7th Jan 22
Stock Market 2022 Requires Different Strategies For Traders/Investors - 7th Jan 22
Old Man Winter Will Stimulate Natural Gas and Heating Oil Demand - 7th Jan 22
Is The Lazy Stock Market Bull Strategy Worth Considering? - 7th Jan 22
METAVERSE - NEW LIFE FOR SONY AGEING GAMING GIANT? - 6th Jan 2022
What Elliott Waves Show for Asia Pacific Stock and Financial Markets 2022 - 6th Jan 2022
Why You Should Register Your Company - 6th Jan 2022
4 Ways to Invest in Silver for 2022 - 6th Jan 2022
UNITY (U) - Metaverse Stock Analysis Investing for 2022 and Beyond - 5th Jan 2022
Stock Market Staving Off Risk-Off - 5th Jan 2022
Gold and Silver Still Hungover After New Year’s Eve - 5th Jan 2022
S&P 500 In an Uncharted Territory, But Is Sky the Limit? - 5th Jan 2022

Market Oracle FREE Newsletter

How to Protect your Wealth by Investing in AI Tech Stocks

Dow Stock Market Trend Forecasting Neural Nets Pattern Recognition

Stock-Markets / AI Sep 15, 2021 - 10:33 PM GMT

By: Nadeem_Walayat

Stock-Markets

So far all attempts to predict the Dow via machine learning have failed, as mentioned above feeding nets with obvious data such as open, close, high, low and then a candle chart version does not work. In fact the neural net part is the easy part via the likes of Tensor Flow. The problem is with the data which as I explained earlier needs to be preprocessed i.e. feeding raw data into the networks as inputs ends up with noisy networks, if it did work then it would be easy to successfully train neural nets and they would be widespread instead as far as I am aware there aren't really any neural nets out there that can successfully trade stocks.

So what does preprocessing actually mean?

There are 2 ways to go about preprocessing data for stock market forecasts.

1. Pattern recognition

2. Trend Analysis preprocessing.


1. Pattern Recognition

One only needs to look at the deep fake videos out there of what actually needs to be done in terms of preprocessing of IMAGES, creating a dataset of hundreds of preprocessed Images of market price charts, say covering a fixed 60 data points (days) each with the goal of predict the next 20 or so days, and there is no guarantee it will work though appears to be the best route to go down, given the success rate of image recognition algorithms, though it would result in a huge neural nets for instance a chart image with a resolution of 256 by 256 pixels would require at least 65,636 input layers! i .e. as the images are broken up and fed to the network

So obviously most of the work, perhaps 95% is in generating the dataset of market chart images where 150 years of Dow data at 60 data points (days) per image stepped every 20 days translates into about 1500 images. Would that be enough? Maybe it could work for a trained / labeled neural net but likely not be enough data for deep learning, the holy grail of machine learning.

So the problem with forecasting markets using neural nets is not with the neural nets, that's the easy bit, but rather creating the dataset's, a lot of time consuming trial and error and I suspect I am going to have to model the neural nets on what I actually do when analysing markets i.e. looking it on varying time frames, zooming in and out.

Likely 60 day data set will not be enough, so require separate neural nets trained with 120 day, and 230 day (1 year) price chart images etc. That acts as preprocessed inputs to the next network, will that work?

Maybe I will need to add nuances by creating dataset's i.e instead of candle charts use swing charts, maybe point and figure etc. Maybe separate neural nets on each dataset that feeds its output as inputs into the next neural net.

2. Trend Analysis Preprocessing

Along with image recognition there is going down the EC route i.e. full spectrum trend analysis, which effectively would mean reinventing the wheel, writing thousands of lines of code in attempts to convert price data into trend analysis data points which basically means creating a whole host of of expert systems that pre-analyse the price data before it is fed into neural nets that along the lines with the EC indicator would each take a huge amount of fine turning, and likely end up with separate useful expert systems in their own right, I.e. trend analysis such as price patterns, support resistance, trend lines, MACD, seasonality, elliot waves. I say reinventing the wheel because I've already done all of this before and more from the Mid 1990's to the early 2000's when I created a multitude of market expert systems and automated technical trading tools, in fact I even had a primitive AI neural net attempting to LEARN from preprocessed data though of course one cannot compare then with now in terms of machine learning as we now have -

a. Infinitely greater compute power! There is no comparison between what one can do with a high end system today than one from 2000.

b. That using neural networks today's is a doddle! About a million times easier than in 2000! In 2000 one only had snippets of information i.e. layers of neurons all connected together by weights was about the extent of information available with much yet to be discovered / invented over the next 20 years.

So I have literally been there and DONE that over a period of about 7 years from around 1994 to 2001, creating what was probably at the time one of the worlds most sophisticated Trend Analysis development environments that I called "TA Dev", literally decades ahead of its time where I included everything I could think of during those 7 years such as Chart pattern recognition,, swing Trader, a primitive neural network, Elliott Waves generator,, seasonal analysis, trading systems galore, and the requisite full spectrum charting etc. But what it lacked at the time was PROCESSING POWER and probably also MEMORY to hold all of the data and variables so was very resources hungry.

A short 3 min video of running the program 20 years later. https://youtu.be/b68yZGc7AmA

The program was good for testing strategies and observations, conducting studies and pattern recognition but was overkill for what was needed to actually trade. During that time period I basically did TA to death! The program was written in Borland Delphi (pascal), which died a slow death during the early 2000's with the last iteration being Delphi 7 (2002) though the last version I actually used was Delphi 5. After Delphi 7 it all become a bit of a mish mash mess, a series of incompatible stop start nonsense that I will now need to sort through to see what could actually work as I attempt to resurrect 'Ta Dev" to use towards generating data for inputs to neural nets.

The last compiled executable of the trading development environment is dated 2nd October 2001 which I managed to get to run under Windows 10 under Window XP service pack 3 compatibility mode, but as has been the case for well over a decade the program fails to load any data into memory probably due to deprecated DLL's that the program is attempting to call.

So the first step is to get an Delphi IDE up and running and then get the code working so that it loads data. If this turns out to be impossible under Windows 10 then an XP system will need to be built.

The twin goals of the software will be to convert the various automated generators to provide preprocessed data for the neural net, and write new code to generate standardised charts for image pattern recognition, that should be straight forward to generate many hundreds of images of charts at the click of a button.

This article is an excerpt form my recent in-depth analysis that contains AI stock 3 year valuation forecasts and 3 high risk reward chinese stocks for the next 5 years. So for immediate first access to ALL of my analysis and trend forecasts then do consider becoming a Patron by supporting my work for just $3 per month that is soon set to increase to $4 per month for new Patrons, so a short window of opportunity exists to lock in at $3 per month. https://www.patreon.com/Nadeem_Walayat.

AI Predicts AI Tech Stock Price Valuations into 2024, Time to Buy Chinese Tech Stocks?

Contents:

  • AI Stocks Value Forecaster (ASVF).
  • How I Use ASVF6 - Percent Upwards Pressure (PUP)
  • AI Stocks Buying Levels Plus ASVF & PUP 
  • AI Stocks Portfolio Buying Levels
  • Dow Stock Market Trend Forecasting Neural Nets
  • Pattern Recognition 
  • Trend Analysis Preprocessing
  • Crossing the Rubicon With These Three High Risk Tech Stocks
  • Cheap Chinese Tech Stock 1
  • Cheap Chinese Tech Stock 2
  • Cheap Chinese Tech Stock 3
  • CME Black Swan

The rest of this extensive analysis has first made available to Patrons who support my work. So for immediate first access to ALL of my analysis and trend forecasts then do consider becoming a Patron by supporting my work for just $3 per month that is soon set to increase to $4 per month for new Patrons, so a short window of opportunity exists to lock in at $3 per month now before the price hike. https://www.patreon.com/Nadeem_Walayat.

Including access to my recent extensive analysis updating AI stocks buying levels as we head towards the window for a significant correction even a possible stock market crash.

AI Stocks Portfolio Buying and Selling Levels, Bubble Valuations 2000 vs 2021

  • Stock Market Bubble Valuations 2000 vs 2021
  • Microsoft to the Moon - OUCH!
  • CISCO to the Moon - OUCH!
  • INTEL to the Moon - OUCH!
  • Tech Stocks in a Bubble today?
  • China / US Stock Markets Divergence
  • AI Stocks Portfolio Buying and SELLING Levels
  • AI Stocks Portfolio Buy / Sell Table Update
  • High Risk Stocks
  • Market Oracle AI Coin Mothballed
  • Global Warming Code RED

Also access to my recent extensive analysis -

Chasing Value with Five More Biotech Stocks for the Long-run

Contents

  • RISK RATINGS
  • HIGH RISK STOCK BUYING LEVELS\
  • Bxxxxxxxxx - Bxxxx -- Risk 3
  • Cxxxxxxxxxxxx- Cxxx - - Risk 5
  • Txxxxxxxxxx - Txx - - Risk 1
  • Bxxxxxxxxxxxxx- Bxxx - Risk 8
  • Axxxxxxxxxxx - Axxx -- Risk 10
  • High Risk Stocks Portfolio Buying Levels
  • Netflix - FAANG a Buy, Sell or Hold?
  • Trending towards Hyperinflation!
  • Delta Variant!
  • Solar CME MULTIPLE Black Swans

And Investing in a Bubble Mania Stock Market Trending Towards Financial Crisis 2.0 CRASH!

  • You Don't Know How Big of a Bubble Your in until AFTER it BURSTS
  • Stock Market Summer Correction
  • REPO Market Brewing Financial Crisis Black Swan Danger
  • Margin Debt Bubble
  • US Bond Market Long-term Trend
  • Michael "Big Short" Burry CRASH and HYPERINFLATION WARNING!
  • Michael Burry's Track Record
  • Michael Burry's Portfolio
  • Investing During Uncertainty
  • AI Stocks Portfolio Buying July Levels Update
  • HEDGING AI Stocks Portfolio
  • Crypto Bear Market Accumulation State
  • Bitcoin Bull / Bear Indicator
  • Market Oracle AI Coin Thoughts
  • Biotech Brief

My analysis schedule includes:

  • Stock Market Trend Forecast September to December 2021
  • Silver Price Trend Analysis, AI Stocks Portfolio Update - 50% Done
  • How to Get Rich! - 90% done - This is a good 6 month work in progress nearing completion.

  • UK House Prices Trend Analysis, including an update for the US and a quick look at Canada - 15% done
  • US Dollar and British Pound analysis
  • Gold Price Trend Analysis - 10%

Again for immediate access to all my work do consider becoming a Patron by supporting my work for just $3 per month that is soon set to increase to $4 per month for new Patrons, so a short window of opportunity exists to lock in at $3 per month. https://www.patreon.com/Nadeem_Walayat.

And ensure you are subscribed to my ALWAYS FREE newsletter for my next in-depth analysis.

Your AI data preprocessing analyst.

Nadeem Walayat

http://www.marketoracle.co.uk

Copyright © 2005-2021 Marketoracle.co.uk (Market Oracle Ltd). All rights reserved.

Nadeem Walayat has over 30 years experience of trading derivatives, portfolio management and analysing the financial markets, including one of few who both anticipated and Beat the 1987 Crash. Nadeem's forward looking analysis focuses on UK inflation, economy, interest rates and housing market. He is the author of five ebook's in the The Inflation Mega-Trend and Stocks Stealth Bull Market series that can be downloaded for Free.

Housing Markets Forecast 2014-2018The Stocks Stealth Bull Market 2013 and Beyond EbookThe Stocks Stealth Bull Market Update 2011 EbookThe Interest Rate Mega-Trend EbookThe Inflation Mega-trend Ebook

Nadeem is the Editor of The Market Oracle, a FREE Daily Financial Markets Analysis & Forecasting online publication that presents in-depth analysis from over 1000 experienced analysts on a range of views of the probable direction of the financial markets, thus enabling our readers to arrive at an informed opinion on future market direction. http://www.marketoracle.co.uk

Disclaimer: The above is a matter of opinion provided for general information purposes only and is not intended as investment advice. Information and analysis above are derived from sources and utilising methods believed to be reliable, but we cannot accept responsibility for any trading losses you may incur as a result of this analysis. Individuals should consult with their personal financial advisors before engaging in any trading activities.

Nadeem Walayat Archive

© 2005-2019 http://www.MarketOracle.co.uk - The Market Oracle is a FREE Daily Financial Markets Analysis & Forecasting online publication.


Post Comment

Only logged in users are allowed to post comments. Register/ Log in