“AI-assisted FA” or “Living the dream”

Sherpa experiments with AI-assisted fundamental analysis using OpenAI’s Playground. This week’s offering is a bit more light hearted.

For those of you who follow me, you might be aware that lately I’ve been deep inside the “ai” rabbit hole, and so this week I decided to take a break from all the “NFT” stuff, and show folks a little experimenting in OpenAI’s playground. You’ll have to register in order to try anything, but the free credits they give you actually go quite a long way.

I’m not going to go into a full ‘tutorial’, as there are a ton available, and I’d like for this week’s offering to be a bit more light-hearted, and get back to the fun of it all. That’s the best part of the space, anyway, where we’re trying new things just to see what happens.

Disclaimer: Do NOT run with any of the code produced by ai as if it’s going to work. It might! It might not! Same with anything else the ai spits out; grain of salt. Let’s just have some fun kicking it around a bit.

Having Some Fun

The first thing I wanted to throw at ai was Solidity. I knew there were more than enough repos & examples of the code lying around that davinci (the most advanced openai model) probably wouldn’t have too much trouble piecing something together.

I was right! Boy, howdy, was I ever.



What happens when we try to make it do a little more? In some cases, it simply pulled the repo of another contract & called it a day. Well…that’s not too far off from how most projects would do it. In other cases? It’s pretty startling just how close it gets to a functional contract.

Okay, so maybe we could reverse this? Use it to find out what a contract does? It’s mostly about writing the initial prompt for the ai, and choosing the right model & settings for your purpose. They even have a number of presets for different uses, such as a Q&A or an ELI5.



That’s actually pretty close to what we’re going to try to do, next. Basically I’m going to search for some contract code and see if I can prompt the AI to tell us what it does.



As it turns out, I’d run into some malicious code not too long ago, for a supposed ‘MEV frontrun bot’ that contained no arbitrage functionality at all; instead, when I tried it on testnet, it sent all the ETH in the contract to another address.



Quick pause here to say: never try these, and even if you know what you’re doing, only deploy on testnet/with a fresh wallet. Better yet, just leave that stuff to folks who like to live dangerously. There is no ‘easy edge’. If there were, they’d just run the contract themselves.

PeckShield & others disclosed similar contracts going around, all with the same basic functionality: when you ‘start’ the contract, thinking that it’s going to begin botting, it immediately sends everything in the contract to an address that you didn’t specify.

I was actually REALLY curious how they managed this, and while it took more than a few tries & rewording the prompts in order to get a sensible result, the ai actually did take a fairly complex contract & spit out the exploit, although I had to edit it down significantly (removing dev comments, etc) to make it fit the OpenAI Playground.

After maybe three or four attempts, it actually did give me a decent answer. Ask your friendliest dev for confirmation, but I think davinci’s nailed it. I mean, more or less. The contract goes through a roundabout way of querying the mempool in order to get the specific address they want to withdraw funds to, when you hit the ‘start’ function on the contract.

This means that even if you’re semi-familiar with Solidity, you might not have seen the hook. But ai did…eventually. When I told it to look for a hook. It also mis-stated some of the syntax, here, as it wouldn’t be going to the ‘contract creator’s’ address, but rather the one it just got from the mempool.



Still, it did a lot better than I could’ve without some digging around. It’s another tool for our fundamental analysis tool belt, and, honestly – a really fun one to just try out a bit. That’s why it’s called a “Playground”.



Until next week, why not try out OpenAI’s Playground and let me know what kind of experiment you come up with on Twitter?