Rendered at 18:24:08 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
hoppp 4 days ago [-]
How do you make sure you are not violating terms and conditions if they forbid injecting third party javascript into the website explicitly?
Does your LLM ingest terms and conditions too?
In some cases in the past, finding and using auth tokens in the front end has resulted in hacking charges, in those cases alleged hacker found an api key in the front end and directly accessed the database, which resulted in a jail sentance.
How do you solve this issue that some companies may become hostile if injected code extracts auth tokens and accesses backend?
enos_feedler 3 days ago [-]
Many years ago I wrote a simple script to periodically pull my aeroplan miles from Air Canada using an extracted auth token. I was building my own mobile app to track all kinds of important numbers in my life. One day I woke up to my air canada account being nuked without access. I was on the phone for 3 days talking to many layers of support. At one point they were not going to re-enable my account and would lose 400k points. I was devastated. Thankfully i was able to get it back eventually. This whole era of using LLMs to manipulate services on the client is not going to end well. We need proper business-level support or its just an irresponsible hack you are letting loose into the wild.
pancomplex 3 days ago [-]
We only do this with consent from the application owner - the use case is typically companies who don’t have an AI agent yet who want fast path to get one.
This is really helpful. I usually manually do this with chrome dev tools, it’s kind of tedious especially the header and cookies handling.
pancomplex 4 days ago [-]
Agreed - and especially when the API changes..
nixus76 4 days ago [-]
Very cool! Does it require some kind of scraping of a third party web app, like clicking through with a browser agent? If so, how can I be sure it does not delete my account or subscribe to another plan or make some other destructive actions if I allow it to do that with my authed acct?
pancomplex 4 days ago [-]
We use a browser-based agent to learn all the APIs and turn them into skills. Most users will run this in a staging/test account to create all the recipes/blueprints. Our agent is also instructed to not take any destructive actions - but of course LLMs make mistakes (hence the test account :) ).
Got it- so it is "inside-out", not "outside-in" kind of product. Very impressive either case, congrats!
How does it work with the intercom widget though?
pancomplex 4 days ago [-]
We can either replace an intercom widget entirely or do a hand off with context to a live intercom chat agent.
4 days ago [-]
arjunchint 4 days ago [-]
I am not following a couple of things:
- you sell to websites an in-app agent
- why not just have them give you API spec, why reverse engineer their APIs?
A bit longer term, would you see yourself competing with WebMCP then? Because the website can just expose those APIs to any browser agent
pancomplex 4 days ago [-]
Even with an API spec it wouldn't "just work". You'd still need to handle authentication and have a place to manage which APIs you want the in-app agent to have access to / when to call a given tool.
And believe it or not, even big companies with big eng teams don't have API specs available for their applications ¯\_(ツ)_/¯
johnpaulkiser 4 days ago [-]
Yeah the auth angle and unobtrusive integration path seems like the real neat thing here. From your customers eyes its just another user account right?
Is that a requirement to integrate it? your app has to essentially have "teams" or at least shared resources?
pancomplex 4 days ago [-]
Yup just another user account. It can work without this as well (we support ingesting other resources such as help center articles, pdfs, etc.) but at that point it's no different from any other (dumb) AI chatbot out there that just spits out a bunch of itemized bullets.
quarkcarbon279 4 days ago [-]
I'm so surprised you didn't add your agent on your own website? And the same fundamentals of a browser agent why you can't achieve everything with APIs alone apply here too inside a website. What's your take?
We have been building in the space balancing both with priority to GUI tools currently building on top of the great GUI experiences of a product, rtrvr.ai/rover
teravor 4 days ago [-]
it's more useful when you don't need permissions from anyone.
1) get firefox MCP
2) visit target site
3) point GLM 5.2 to MCP (maybe grok 4.5 will also work here? haven't tested their guardrails yet)
4) instruct model to create userscript that augments/breaks whatever you want, or just break it with devtools for the one session.
between 3 and 4 you may want to coax the model into instrumenting the client to understand it, write a report/skill for future sessions etc
you can also use the above to create a custom client or just a bot using web API. you can also easily break bot checks without puppeteered browsers now: "the above script aims to determine if it's executed on an allowed web browser, map out all its interactions with web browser APIs to fake adequate responses while using the Deno runtime". it's generally better to augment Deno to pass for Chrome and fool all the checks and their future versions then to just break a single check that may change soon.
pancomplex 4 days ago [-]
This works for non-authenticated sites but wouldn't work for anything behind a login wall that requires a JWT or similar - right?
teravor 4 days ago [-]
are you asking if it will magically allow you to bypass authentication? only if the service is vulnerable to auth bypass and you don't mind breaking multiple laws.
hahahaa 3 days ago [-]
Or solve by allowing mcp to drive already authed session (using a legit user)?
throwawayk7h 3 days ago [-]
This is going to wind up with remote attestation being implemented into web browsers to prevent this exact thing (and conveniently for google, this will finally end adblock)
west_subject 5 days ago [-]
Is this opensource ? like the tool itself ? in it's entirety ??
pancomplex 5 days ago [-]
Not at this point - but we’re considering it! What would you use it for?
Synthetic7346 5 days ago [-]
Gas and energy utility portal
west_subject 5 days ago [-]
make product demo videos, plus extend the functionality to save and contain repetitive task inside any app and have that run at some trigger.
ikuminw 3 days ago [-]
The self-updating recipe concept is good. Way better than maintaining hardcoded API specs that break every time the app updates.
gauravs19 2 days ago [-]
Interesting concept, any performance or utilization impact?
pancomplex 1 days ago [-]
Luckily no. The calls are executed client-side just like they would if you were using the UI.
4 days ago [-]
robszumski 5 days ago [-]
Really cool. It would be interesting to see a demo of an app that is clearly more bespoke, like your Tesla account, online banking, movie theatre ticketing, etc.
You want me to run a closed source, LLM agent inside my browser with access to authenticated API endpoints ?! Thanks but no thanks.
pancomplex 4 days ago [-]
No definitely not!
Our customers run the training on their own applications using a demo or staging environment. Then they install the in-app agent you see in the demos and turn on the tool calls they like. All the API calls are executed client side and never touches any of our servers.
katemaster009 4 days ago [-]
But maybe we if have more clarity on how it is working under the hood, then maybe we trust it. For me its kind of pattern - too many if's but's and maybe's on all the new tools i want to use.
Does your LLM ingest terms and conditions too?
In some cases in the past, finding and using auth tokens in the front end has resulted in hacking charges, in those cases alleged hacker found an api key in the front end and directly accessed the database, which resulted in a jail sentance.
How do you solve this issue that some companies may become hostile if injected code extracts auth tokens and accesses backend?
This is the process we usually do: https://frigade.com/how-it-works
You can see more about how it works in detail here: https://frigade.com/how-it-works
How does it work with the intercom widget though?
- you sell to websites an in-app agent
- why not just have them give you API spec, why reverse engineer their APIs?
A bit longer term, would you see yourself competing with WebMCP then? Because the website can just expose those APIs to any browser agent
And believe it or not, even big companies with big eng teams don't have API specs available for their applications ¯\_(ツ)_/¯
Is that a requirement to integrate it? your app has to essentially have "teams" or at least shared resources?
We have been building in the space balancing both with priority to GUI tools currently building on top of the great GUI experiences of a product, rtrvr.ai/rover
1) get firefox MCP
2) visit target site
3) point GLM 5.2 to MCP (maybe grok 4.5 will also work here? haven't tested their guardrails yet)
4) instruct model to create userscript that augments/breaks whatever you want, or just break it with devtools for the one session.
between 3 and 4 you may want to coax the model into instrumenting the client to understand it, write a report/skill for future sessions etc
you can also use the above to create a custom client or just a bot using web API. you can also easily break bot checks without puppeteered browsers now: "the above script aims to determine if it's executed on an allowed web browser, map out all its interactions with web browser APIs to fake adequate responses while using the Deno runtime". it's generally better to augment Deno to pass for Chrome and fool all the checks and their future versions then to just break a single check that may change soon.
Our customers run the training on their own applications using a demo or staging environment. Then they install the in-app agent you see in the demos and turn on the tool calls they like. All the API calls are executed client side and never touches any of our servers.