> ## Content Index
> Fetch the complete content index at: https://www.smdp.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Wall Street is betting big on AI, California consumers must be protected
- URL: https://www.smdp.com/wall-street-is-betting-big-on-ai-california-consumers-must-be-protected/
- Published: 2024-03-29T13:00:00.000Z
- Updated: 2024-12-04T15:20:52.000Z
- Author: Guest Author
- Tags: Opinion, AI. California, wall street, #wp, #wp-post, #Import 2025-11-06 12:20

Artificial intelligence may soon replace financial advisors, and that’s just the start. While one algorithm predicts stock price, another decides what to disclose to regulators. Chat GPT-style language models could soon [hallucinate](https://cloud.google.com/discover/what-are-ai-hallucinations?ref=smdp.com#section-1) your mortgage application or life savings, and you won’t even know it.

I spent several months combing through the US Patent and Trademark Office to see how the world’s most powerful investment banks are [incorporating](https://www.bloomberg.com/news/features/2023-05-31/jpmorgan-s-push-into-finance-ai-has-wall-street-rushing-to-catch-up?ref=smdp.com) artificial intelligence, and it’s a scary glimpse into the future.

JPMorgan has applied for a trademark called “Index GPT” that would give [financial](https://www.cnbc.com/2023/05/25/jpmorgan-develops-ai-investment-advisor.html?ref=smdp.com) advice, and one that would match companies with investors. Goldman Sachs is seeking to patent AI that will combine all the data a trader would need to predict stock prices, and another to predict a hedging portfolio. Already, ING Group is screening for potential defaulters.

There is even AI to translate “[Fedspeak](https://www.bloomberg.com/news/articles/2005-03-23/a-laymans-guide-to-fedspeak?ref=smdp.com)” so banks can tell if statements by regulators are “dovish” or “hawkish.”

Banks are pouring billions of dollars into AI research, patents and financing without adequate safeguards. A lot of attention is devoted to tech companies developing artificial intelligence, but Wall Street banks are just as interested. Financial services spending on AI is larger than any other industry, [exceeding](https://www.technologyreview.com/2022/09/20/1059630/cio-vision-2025-bridging-the-gap-between-bi-and-ai/?ref=smdp.com) even the tech industry.

There is still little we know about generative AI. Even engineers and coders don’t [understand](https://news.utexas.edu/2023/05/01/brain-activity-decoder-can-reveal-stories-in-peoples-minds/?ref=smdp.com) how AI works. Unlike previous models, OpenAI decided not to disclose the training data of GPT-4\. ChatGPT has scraped the Internet for over 300 billion words, and there are concerns that it has sucked up personal information along the way. In addition, [OpenAI](https://openai.com/policies/privacy-policy?ref=smdp.com) says its language model has a “high risk of economic harm” due to a “tendency to hallucinate,” and should come with “a disclaimer.”

Despite these concerns, JPMorgan [said](https://reports.jpmorganchase.com/investor-relations/2022/ar-ceo-letters.htm?ref=smdp.com) that data and AI will be “critical” to the company’s future success. It currently has more than 300 use cases for AI in production.

But the lack of transparency surrounding AI and its potential [for bias](https://wired.com/story/efforts-make-text-ai-less-racist-terrible/?ref=smdp.com) means mysterious AI could push risky investments and loans, or hallucinate bad advice on managing debt without a consumer even knowing it was AI.

Without sound regulation, the next financial crisis could be caused by AI, igniting in the mortgage or equity market due to a handful of banks relying on the same algorithms. Don’t just take it from me. Much smarter people like Gary Gensler, chair of the US Securities and Exchange Commission, predicted that within 10 years AI will be [responsible](https://www.ft.com/content/8227636f-e819-443a-aeba-c8237f0ec1ac?ref=smdp.com) for some sort of financial crisis.

The major concerns are algorithmic complexity, a lack of transparency and biased or false information. For example, Goldman’s AI could be used to create and price out a new type of derivative. These complicated financial instruments enabled Wall Street to gamble with billions of dollars on the housing market. As a result, 6 million Americans lost their [homes](https://smdp.com/2024/03/09/california-county-is-testing-ai-to-prevent-homelessness/?ref=smdp.com) during the Great Recession.

How else can AI impact our lives financially? AI can also start to “drift,” meaning it can deviate from its intended use and perpetuate bias. An AI could start thinking a certain race or address equals bad credit and deny loans based on that. This has already [happened](https://themarkup.org/denied/2021/08/25/the-secret-bias-hidden-in-mortgage-approval-algorithms?ref=smdp.com).

Thankfully, California is trying to rein in AI. Right now the California Privacy Protection Agency is [drafting](https://cppa.ca.gov/announcements/2023/20231127.html?ref=smdp.com) automated decision-making rules under the California Consumer Privacy Act, which would [safeguard](https://www.insideprivacy.com/ccpa/california-privacy-protection-agency-takes-next-step-on-new-automated-decision-making-regulations-and-privacy-risk-assessments/?ref=smdp.com) against AI’s risks and biases regarding personal data. For example, a business would have to tell consumers that it uses automated decision-making, details about the algorithm’s logic and give a chance to opt out of the decision, especially if it involves finances.

And for the first time, there are rules dedicated to [generative](https://calmatters.org/economy/technology/2024/03/california-ai-purchasing-guidelines/?ref=smdp.com) AI and training data. Businesses using language models will have to disclose if those models use personal information to train AI, and consumers will be able to opt out.

America is falling behind on AI regulation. It cannot be slow to act like it was regarding social media when both sound regulation and innovation are possible.

*Justin Kloczko is a tech and privacy advocate for Consumer Watchdog*