13
Excellent for local models running via OpenCode (github.com)
submitted 3 days ago by to c/localllama@sh.itjust.works
20 comments fedilink hide all child comments
 

Buddy: an open source app that lets AI agents (Claude Code, Codex, OpenCode) use your Android phone

I've been building Buddy, a free, open source (Apache 2.0) Android app. You type or say what you want in a floating bubble, and an AI agent reads the screen, taps and types through your apps to do it.

  • Your choice of agent: Claude Code, Codex, OpenCode, Antigravity or Grok, running inside the app (Alpine Linux via proot) on your own plan. OpenCode's free models need no account.
  • No root, adb or PC: it reads the screen and taps through the accessibility service.
  • Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.
  • Builds apps: with Linux tools on, it builds real APKs on the phone. This week I drew a maze on a paper towel, took a photo, and it made it a tilt game. The first build crashed on launch; it found the bug and fixed it.

To be upfront:

  • What the agent reads on your screen goes to the AI service you pick.
  • It's early (2.2), tested most on a OnePlus 12 with Android 16.
  • It's not on Play (Google rejected the accessibility use), so it's an APK from GitHub releases, built by GitHub Actions from the source.

Source and releases: github.com/ghorbelhamdi/buddy-android

Happy to answer questions here.

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 days ago (3 children)
  • Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.

Lol, sure it does. How many times have wee seen this claim, now?

  • permalink
  • fedilink
  • source
  • hideshow 3 child comments
    •
  • [–] 3 points 3 days ago (2 children)

    Fair to be skeptical, but it isn't the model being asked nicely. Buddy checks every tap itself: when the button is Send, Post, Buy, Pay, Delete and similar (by its label or id, in several languages), Buddy stops and shows an approval card, whatever the model wants. It's a word list, so an oddly named button can slip through; it's all in Guard.java if you want to look: https://github.com/ghorbelhamdi/buddy-android/blob/main/app/src/main/java/app/buddy/assistant/Guard.java

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 days ago (1 child)

    When I see word matching like this, in the context of AI, I still fear it's embedded in text prompt interfaces, meaning it can be evaded. (D E L E T E instead of delete etc)

    It also doesn't tell me much about the wider risks and gaps of the connected system architecture - which, to be fair, I haven't inspected or analyzed, so I don't know one way or the other. One guard may be better than none, but doesn't define or fully describe overall/system risk.

    Naturally, I'm more risk-averse and skeptical than many other people. I guess the original commenter is, too. :)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 days ago

    It doesn't match the model's text. The check runs in the accessibility service on the actual button the tap would land on: its label, icon description and view id, as the app reports them. The model can write D E L E T E all it wants, the button still says Delete. It also can't tap Buddy's own approval card to approve itself.

    Where it can miss: a button with an odd or blank label, and apps you mark as trusted, where asking is left to the agent. And agreed, it's one guard, not the whole risk picture. Prompt injection from what's on screen is the bigger open problem, and the approval card is the backstop there.

  • permalink
  • fedilink
  • source
  • parent