Best of the Week
Most Popular
1. TESLA! Cathy Wood ARK Funds Bubble BURSTS! - 12th May 21
2.Stock Market Entering Early Summer Correction Trend Forecast - 10th May 21
3.GOLD GDX, HUI Stocks - Will Paradise Turn into a Dystopia? - 11th May 21
4.Crypto Bubble Bursts! Nicehash Suspends Coinbase Withdrawals, Bitcoin, Ethereum Bear Market Begins - 16th May 21
5.Crypto Bubble BURSTS! BTC, ETH, XRP CRASH! NiceHash Seizes Funds on Account Halting ALL Withdrawals! - 19th May 21
6.Cathy Wood Ark Invest Funds Bubble BURSTS! ARKK, ARKG, Tesla Entering Severe Bear Market - 13th May 21
7.Stock Market - Should You Be In Cash Right Now? - 17th May 21
8.Gold to Benefit from Mounting US Debt Pile - 14th May 21
9.Coronavius Covid-19 in Italy in August 2019! - 13th May 21
10.How to Invest in HIGH RISK Tech Stocks for 2021 and Beyond - Part 2 of 2 - 18th May 21
Last 7 days
US Economy and Stock Market Addicted to Deficit Spending - 17th Oct 21
The Gold Price And Inflation - 17th Oct 21
Went Long the Crude Oil? Beware of the Headwinds Ahead… - 17th Oct 21
Watch These Next-gen Cloud Computing Stocks - 17th Oct 21
Overclockers UK Custom Built PC 1 YEAR Use Review Verdict - Does it Still Work? - 16th Oct 21
Altonville Mine Tours Maze at Alton Towers Scarefest 2021 - 16th Oct 21
How to Protect Your Self From a Stock Market CRASH / Bear Market? - 14th Oct 21
The Only way to Crush Inflation (not stocks) - 14th Oct 21
Why "Losses Are the Norm" in the Stock Market - 14th Oct 21
Sub Species Castle Maze at Alton Towers Scarefest 2021 - 14th Oct 21
Which Wallet is Best for Storing NFTs? - 14th Oct 21
Ailing UK Pound Has Global Effects - 14th Oct 21
How to Get 6 Years Life Out of Your Overclocked PC System, Optimum GPU, CPU and MB Performance - 13th Oct 21
The Demand Shock of 2022 - 12th Oct 21
4 Reasons Why NFTs Could Be The Future - 12th Oct 21
Crimex Silver: Murder Most Foul - 12th Oct 21
Bitcoin Rockets In Preparation For Liftoff To $100,000 - 12th Oct 21
INTEL Tech Stock to the MOON! INTC 2000 vs 2021 Market Bubble WARNING - 11th Oct 21
AI Stocks Portfolio Buying and Selling Levels Going Into Market Correction - 11th Oct 21
Stock Market Wall of Worry Meets NFPs - 11th Oct 21
Stock Market Intermediate Correction Continues - 11th Oct 21
China / US Stock Markets Divergence - 10th Oct 21
Can US Save Taiwan From China? Taiwan Strait Naval Battle - PLA vs 7th Fleet War Game Simulation - 10th Oct 21
Gold Price Outlook: The Inflation Chasm Between Europe and the US - 10th Oct 21
US Real Estate ETFs React To Rising Housing Market Mortgage Interest Rates - 10th Oct 21
US China War over Taiwan Simulation 2021, Invasion Forecast - Who Will Win? - 9th Oct 21
When Will the Fed Taper? - 9th Oct 21
Dancing with Ghouls and Ghosts at Alton Towers Scarefest 2021 - 9th Oct 21
Stock Market FOMO Going into Crash Season - 8th Oct 21
Scan Computers - Custom Build PC 6 Months Later, Reliability, Issues, Quality of Tech Support Review - 8th Oct 21
Gold and Silver: Your Financial Main Battle Tanks - 8th Oct 21
How to handle the “Twin Crises” Evergrande and Debt Ceiling Threatening Stocks - 8th Oct 21
Why a Peak in US Home Prices May Be Approaching - 8th Oct 21
Alton Towers Scarefest is BACK! Post Pandemic Frights Begin, What it's Like to Enter Scarefest 2021 - 8th Oct 21
AJ Bell vs II Interactive Investor - Which Platform is Best for Buying US FAANG Stocks UK Investing - 7th Oct 21
Gold: Evergrande Investors' Savior - 7th Oct 21
Here's What Really Sets Interest Rates (Not Central Banks) - 7th Oct 21
CISCO 2020 Dot com Bubble Stock vs 2021 Bubble Tech Stocks Warning Analysis - 6th Oct 21
Precious Metals Complex Searching for a Bottom - 6th Oct 21
FB, AMZN, NFLX, GOOG, AAPL and FANG+ '5 Waves' Speaks Volumes - 6th Oct 21
Budgies Flying Ability 10 Weeks After wings Clipped, Flight Feathers Cut Grow Back - 6th Oct 21
Why Silver Price Could Crash by 20%! - 5th Oct 21
Will China's Crackdown Send Bitcoin's Price Tumbling? - 5th Oct 21
Natural Gas News: Europe Lacks Supply, So It Turns to Asia - 5th Oct 21
Stock Market Correction: One More Spark to Light the Fire? - 5th Oct 21
Fractal Design Meshify S2, Best PC Case Review, Build Quality, Airflow etc. - 5th Oct 21
Chasing Value with Five More Biotech Stocks for the Long-run - 4th Oct 21
Gold’s Century - While stocks dominated headlines, gold quietly performed - 4th Oct 21
NASDAQ Stock Market Head-n-Shoulders Warns Of Market Weakness – Critical Topping Pattern - 4th Oct 21
US Dollar on plan, attended by the Gold/Silver ratio - 4th Oct 21
Aptorum Group - APM - High RIsk Biotech Stocks Buy, Sell, Hold Investing Analysis for the Long-run - 3rd Oct 21
US Close to Hitting the Debt Ceiling: Gold Doesn’t Care - 3rd Oct 21
Powell: Inflation Might Not Be Transitory, After All - 3rd Oct 21
Original Oculus VR HeadSet Rift Dev Kit v1 Before Facebook Bought Oculus - 3rd Oct 21
Microsoft Stock Valuation 2021 vs 2000 Bubble - Buy Sell or Hold Invest Analysis - 1st Oct 21
How to profit off the Acquisition spree in Fintech Stocks - 1st Oct 21
�� Halloween 2021 TESCO Shopping Before the Next Big Panic Buying! �� - 1st Oct 2
The Guide to Building a Design Portfolio Online - 1st Oct 21
BioDelivery Sciences International - BDSI - High RIsk Biotech Stocks Buy, Sell, Hold Investing Analysis for the Long-run - 30th Sep 21
America’s Revolving-Door Politics Behind the Fall of US-Sino Ties - 30th Sep 21
Dovish to Hawkish Fed: Sounds Bearish for Gold - 30th Sep 21
Stock Market Gauntlet to the Fed - 30th Sep 21
Should you include ESG investments in your portfolio? - 30th Sep 21
Takeda - TAK - High RIsk Biotech Stocks Buy, Sell, Hold Investing Analysis for the Long-run - 29th Sep 21
Stock Market Wishing Away Inflation - 29th Sep 21
Why Workers Are NOT Returning to Work as Lockdown's End - Wage Slaves Rebellion - 29th Sep 21
UK Fuel PANIC! Fighting at the Petrol Pumps! As Lemmings Create a New Crisis - 29th Sep 21
Gold Could See Tapering as Soon as November! - 29th Sep 21

Market Oracle FREE Newsletter

How to Protect your Wealth by Investing in AI Tech Stocks

Dow Stock Market Trend Forecasting Neural Nets Pattern Recognition

Stock-Markets / AI Sep 15, 2021 - 10:33 PM GMT

By: Nadeem_Walayat

Stock-Markets

So far all attempts to predict the Dow via machine learning have failed, as mentioned above feeding nets with obvious data such as open, close, high, low and then a candle chart version does not work. In fact the neural net part is the easy part via the likes of Tensor Flow. The problem is with the data which as I explained earlier needs to be preprocessed i.e. feeding raw data into the networks as inputs ends up with noisy networks, if it did work then it would be easy to successfully train neural nets and they would be widespread instead as far as I am aware there aren't really any neural nets out there that can successfully trade stocks.

So what does preprocessing actually mean?

There are 2 ways to go about preprocessing data for stock market forecasts.

1. Pattern recognition

2. Trend Analysis preprocessing.


1. Pattern Recognition

One only needs to look at the deep fake videos out there of what actually needs to be done in terms of preprocessing of IMAGES, creating a dataset of hundreds of preprocessed Images of market price charts, say covering a fixed 60 data points (days) each with the goal of predict the next 20 or so days, and there is no guarantee it will work though appears to be the best route to go down, given the success rate of image recognition algorithms, though it would result in a huge neural nets for instance a chart image with a resolution of 256 by 256 pixels would require at least 65,636 input layers! i .e. as the images are broken up and fed to the network

So obviously most of the work, perhaps 95% is in generating the dataset of market chart images where 150 years of Dow data at 60 data points (days) per image stepped every 20 days translates into about 1500 images. Would that be enough? Maybe it could work for a trained / labeled neural net but likely not be enough data for deep learning, the holy grail of machine learning.

So the problem with forecasting markets using neural nets is not with the neural nets, that's the easy bit, but rather creating the dataset's, a lot of time consuming trial and error and I suspect I am going to have to model the neural nets on what I actually do when analysing markets i.e. looking it on varying time frames, zooming in and out.

Likely 60 day data set will not be enough, so require separate neural nets trained with 120 day, and 230 day (1 year) price chart images etc. That acts as preprocessed inputs to the next network, will that work?

Maybe I will need to add nuances by creating dataset's i.e instead of candle charts use swing charts, maybe point and figure etc. Maybe separate neural nets on each dataset that feeds its output as inputs into the next neural net.

2. Trend Analysis Preprocessing

Along with image recognition there is going down the EC route i.e. full spectrum trend analysis, which effectively would mean reinventing the wheel, writing thousands of lines of code in attempts to convert price data into trend analysis data points which basically means creating a whole host of of expert systems that pre-analyse the price data before it is fed into neural nets that along the lines with the EC indicator would each take a huge amount of fine turning, and likely end up with separate useful expert systems in their own right, I.e. trend analysis such as price patterns, support resistance, trend lines, MACD, seasonality, elliot waves. I say reinventing the wheel because I've already done all of this before and more from the Mid 1990's to the early 2000's when I created a multitude of market expert systems and automated technical trading tools, in fact I even had a primitive AI neural net attempting to LEARN from preprocessed data though of course one cannot compare then with now in terms of machine learning as we now have -

a. Infinitely greater compute power! There is no comparison between what one can do with a high end system today than one from 2000.

b. That using neural networks today's is a doddle! About a million times easier than in 2000! In 2000 one only had snippets of information i.e. layers of neurons all connected together by weights was about the extent of information available with much yet to be discovered / invented over the next 20 years.

So I have literally been there and DONE that over a period of about 7 years from around 1994 to 2001, creating what was probably at the time one of the worlds most sophisticated Trend Analysis development environments that I called "TA Dev", literally decades ahead of its time where I included everything I could think of during those 7 years such as Chart pattern recognition,, swing Trader, a primitive neural network, Elliott Waves generator,, seasonal analysis, trading systems galore, and the requisite full spectrum charting etc. But what it lacked at the time was PROCESSING POWER and probably also MEMORY to hold all of the data and variables so was very resources hungry.

A short 3 min video of running the program 20 years later. https://youtu.be/b68yZGc7AmA

The program was good for testing strategies and observations, conducting studies and pattern recognition but was overkill for what was needed to actually trade. During that time period I basically did TA to death! The program was written in Borland Delphi (pascal), which died a slow death during the early 2000's with the last iteration being Delphi 7 (2002) though the last version I actually used was Delphi 5. After Delphi 7 it all become a bit of a mish mash mess, a series of incompatible stop start nonsense that I will now need to sort through to see what could actually work as I attempt to resurrect 'Ta Dev" to use towards generating data for inputs to neural nets.

The last compiled executable of the trading development environment is dated 2nd October 2001 which I managed to get to run under Windows 10 under Window XP service pack 3 compatibility mode, but as has been the case for well over a decade the program fails to load any data into memory probably due to deprecated DLL's that the program is attempting to call.

So the first step is to get an Delphi IDE up and running and then get the code working so that it loads data. If this turns out to be impossible under Windows 10 then an XP system will need to be built.

The twin goals of the software will be to convert the various automated generators to provide preprocessed data for the neural net, and write new code to generate standardised charts for image pattern recognition, that should be straight forward to generate many hundreds of images of charts at the click of a button.

This article is an excerpt form my recent in-depth analysis that contains AI stock 3 year valuation forecasts and 3 high risk reward chinese stocks for the next 5 years. So for immediate first access to ALL of my analysis and trend forecasts then do consider becoming a Patron by supporting my work for just $3 per month that is soon set to increase to $4 per month for new Patrons, so a short window of opportunity exists to lock in at $3 per month. https://www.patreon.com/Nadeem_Walayat.

AI Predicts AI Tech Stock Price Valuations into 2024, Time to Buy Chinese Tech Stocks?

Contents:

  • AI Stocks Value Forecaster (ASVF).
  • How I Use ASVF6 - Percent Upwards Pressure (PUP)
  • AI Stocks Buying Levels Plus ASVF & PUP 
  • AI Stocks Portfolio Buying Levels
  • Dow Stock Market Trend Forecasting Neural Nets
  • Pattern Recognition 
  • Trend Analysis Preprocessing
  • Crossing the Rubicon With These Three High Risk Tech Stocks
  • Cheap Chinese Tech Stock 1
  • Cheap Chinese Tech Stock 2
  • Cheap Chinese Tech Stock 3
  • CME Black Swan

The rest of this extensive analysis has first made available to Patrons who support my work. So for immediate first access to ALL of my analysis and trend forecasts then do consider becoming a Patron by supporting my work for just $3 per month that is soon set to increase to $4 per month for new Patrons, so a short window of opportunity exists to lock in at $3 per month now before the price hike. https://www.patreon.com/Nadeem_Walayat.

Including access to my recent extensive analysis updating AI stocks buying levels as we head towards the window for a significant correction even a possible stock market crash.

AI Stocks Portfolio Buying and Selling Levels, Bubble Valuations 2000 vs 2021

  • Stock Market Bubble Valuations 2000 vs 2021
  • Microsoft to the Moon - OUCH!
  • CISCO to the Moon - OUCH!
  • INTEL to the Moon - OUCH!
  • Tech Stocks in a Bubble today?
  • China / US Stock Markets Divergence
  • AI Stocks Portfolio Buying and SELLING Levels
  • AI Stocks Portfolio Buy / Sell Table Update
  • High Risk Stocks
  • Market Oracle AI Coin Mothballed
  • Global Warming Code RED

Also access to my recent extensive analysis -

Chasing Value with Five More Biotech Stocks for the Long-run

Contents

  • RISK RATINGS
  • HIGH RISK STOCK BUYING LEVELS\
  • Bxxxxxxxxx - Bxxxx -- Risk 3
  • Cxxxxxxxxxxxx- Cxxx - - Risk 5
  • Txxxxxxxxxx - Txx - - Risk 1
  • Bxxxxxxxxxxxxx- Bxxx - Risk 8
  • Axxxxxxxxxxx - Axxx -- Risk 10
  • High Risk Stocks Portfolio Buying Levels
  • Netflix - FAANG a Buy, Sell or Hold?
  • Trending towards Hyperinflation!
  • Delta Variant!
  • Solar CME MULTIPLE Black Swans

And Investing in a Bubble Mania Stock Market Trending Towards Financial Crisis 2.0 CRASH!

  • You Don't Know How Big of a Bubble Your in until AFTER it BURSTS
  • Stock Market Summer Correction
  • REPO Market Brewing Financial Crisis Black Swan Danger
  • Margin Debt Bubble
  • US Bond Market Long-term Trend
  • Michael "Big Short" Burry CRASH and HYPERINFLATION WARNING!
  • Michael Burry's Track Record
  • Michael Burry's Portfolio
  • Investing During Uncertainty
  • AI Stocks Portfolio Buying July Levels Update
  • HEDGING AI Stocks Portfolio
  • Crypto Bear Market Accumulation State
  • Bitcoin Bull / Bear Indicator
  • Market Oracle AI Coin Thoughts
  • Biotech Brief

My analysis schedule includes:

  • Stock Market Trend Forecast September to December 2021
  • Silver Price Trend Analysis, AI Stocks Portfolio Update - 50% Done
  • How to Get Rich! - 90% done - This is a good 6 month work in progress nearing completion.

  • UK House Prices Trend Analysis, including an update for the US and a quick look at Canada - 15% done
  • US Dollar and British Pound analysis
  • Gold Price Trend Analysis - 10%

Again for immediate access to all my work do consider becoming a Patron by supporting my work for just $3 per month that is soon set to increase to $4 per month for new Patrons, so a short window of opportunity exists to lock in at $3 per month. https://www.patreon.com/Nadeem_Walayat.

And ensure you are subscribed to my ALWAYS FREE newsletter for my next in-depth analysis.

Your AI data preprocessing analyst.

Nadeem Walayat

http://www.marketoracle.co.uk

Copyright © 2005-2021 Marketoracle.co.uk (Market Oracle Ltd). All rights reserved.

Nadeem Walayat has over 30 years experience of trading derivatives, portfolio management and analysing the financial markets, including one of few who both anticipated and Beat the 1987 Crash. Nadeem's forward looking analysis focuses on UK inflation, economy, interest rates and housing market. He is the author of five ebook's in the The Inflation Mega-Trend and Stocks Stealth Bull Market series that can be downloaded for Free.

Housing Markets Forecast 2014-2018The Stocks Stealth Bull Market 2013 and Beyond EbookThe Stocks Stealth Bull Market Update 2011 EbookThe Interest Rate Mega-Trend EbookThe Inflation Mega-trend Ebook

Nadeem is the Editor of The Market Oracle, a FREE Daily Financial Markets Analysis & Forecasting online publication that presents in-depth analysis from over 1000 experienced analysts on a range of views of the probable direction of the financial markets, thus enabling our readers to arrive at an informed opinion on future market direction. http://www.marketoracle.co.uk

Disclaimer: The above is a matter of opinion provided for general information purposes only and is not intended as investment advice. Information and analysis above are derived from sources and utilising methods believed to be reliable, but we cannot accept responsibility for any trading losses you may incur as a result of this analysis. Individuals should consult with their personal financial advisors before engaging in any trading activities.

Nadeem Walayat Archive

© 2005-2019 http://www.MarketOracle.co.uk - The Market Oracle is a FREE Daily Financial Markets Analysis & Forecasting online publication.


Post Comment

Only logged in users are allowed to post comments. Register/ Log in