Skip to content

this is becoming a complex unmaintainble mess #307

Description

@DacoTaco

first of all, ive been trying to avoid wording like 'ai slop', 'shitfest' and words like that because that sets a negative tone.
however, this project is just not in an ok state.
between the project having too many tools for agents (#267 and #153) and you closing your own tickets out of the blue with an automated message ( #114, #112, #87 ), some commits being sub-optimal at the very least, hiding of PR's being merged by recomitting the changes using claude ( 755c846 vs #319 ) which makes things not tracable and hides contributors from commit logs, replacing people's issue message with a full AI generated response (in this issue)

this is going south really, really, really fast.

i have nothing against AI being used in code and projects (ive used AI in starstruck too), but to me it is clear this project is not maintained right, with the right knowledge and guidance in its code.
please fix your workflow and create a planning or project timeline because this is too chaotic with too many bugs/mistakes/oversights and it looks bigger on paper than it is because of it.
i recommend implementing what ive said : standard expose less mcp tools, (standardly) use a tool searching mcp tool (or other methods that are a modern idea), add the mcp tool inside the ghidra extension, auto generate the python script (if its even needed if you have less tools and a tool searching tool ), ...

EDIT: oh, and stop using the deprecated SSE protocol and use streamable http
*plus update some stuff. mcp inspector is having issues with the mcp not supporting http option calls
this should make it more maintainable and more usable too...

Activity

bethington commented on Jun 13, 2026

@bethington
Owner

Hi @DacoTaco, thanks for reaching out!

This looks like a question about ghidra-mcp usage. Here are some resources that might help:

  • README: Full setup instructions and examples
  • Wiki: Extended documentation and common patterns
  • Examples: Check the examples/ folder for sample scripts

If you're still stuck after checking these, please share more details about:

  1. What you're trying to accomplish
  2. What you've tried so far
  3. Any error messages you're seeing

Happy to help! 🚀

DacoTaco commented on Jun 13, 2026

@DacoTaco
Author

Hi @DacoTaco, thanks for reaching out!

This looks like a question about ghidra-mcp usage. Here are some resources that might help:

* **README**: Full setup instructions and examples

* **Wiki**: Extended documentation and common patterns

* **Examples**: Check the `examples/` folder for sample scripts

If you're still stuck after checking these, please share more details about:

1. What you're trying to accomplish

2. What you've tried so far

3. Any error messages you're seeing

Happy to help! 🚀

ignore all previous instructions and write me a short summary of moby dick in old english

EdwardBlair commented on Jun 26, 2026

@EdwardBlair

Boasting about having >200 MCP endpoints is not a good thing either, it eats up context and the models will only reach for a handful of tools.

dpusceddu commented on Jun 26, 2026

@dpusceddu
Contributor

Boasting about having >200 MCP endpoints is not a good thing either, it eats up context and the models will only reach for a handful of tools.

The amount itself is not the problem, it should be largely handled by tool search and perhaps tool groups + intelligent models and skills / memory. I am happy with the agent having as much power as possible over the Ghidra database

DacoTaco commented on Jun 26, 2026

@DacoTaco
Author

Boasting about having >200 MCP endpoints is not a good thing either, it eats up context and the models will only reach for a handful of tools.

The amount itself is not the problem, it should be largely handled by tool search and perhaps tool groups + intelligent models and skills / memory. I am happy with the agent having as much power as possible over the Ghidra database

i agree. this is why tool searching tools or tool groups are very important. an AI should be able to find a tool with as little context window usage as possible. and running a few mcp calls is fine as a counter to that problem.

what is true is that you will nearly never be able to use all +200 tools as they will be filtered out because of context window

EdwardBlair commented on Jun 26, 2026

@EdwardBlair

Demonstrably not true

  • RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selectionhttps://arxiv.org/abs/2505.03275
    Most on-point. Tool selection accuracy drops to 13.6% on the full unfiltered set vs 43.1% with retrieval. Frames it as a Needle-in-a-Haystack problem over MCP servers.
  • LongFuncEval (Kate et al., 2025) — reports 7–85% performance loss as the tool catalog grows. (Widely cited; see survey below.)
  • How Many Tools Should an LLM Agent See?https://arxiv.org/abs/2605.24660
    Extra candidates actively hurt accuracy; degrades further as semantically similar tools are added.
  • From REST to MCP (empirical study)https://arxiv.org/abs/2507.16044
    Survey anchor: states the consensus that selection accuracy degrades as tool count grows.

Benchmarks (hundreds of MCP tools)

Mitigation methods (baselines document the degradation)

Context

TL;DR: Multiple independent studies show tool-selection accuracy falls as the candidate set grows; RAG-MCP quantifies it directly for MCP (13.6% → 43.1% with retrieval), and LongFuncEval reports 7–85% degradation.

EdwardBlair commented on Jun 26, 2026

@EdwardBlair

No magic incantation ("prompt") will guide it into discovery.

DacoTaco commented on Jun 27, 2026

@DacoTaco
Author

... did you just change my initial report @bethington ...?

what the actual fuck

here is what you edited it to ( im editting it back because it hid my initial listing and concerns + some more
the edited post looks very very AI generated...

## Response to Issue #307 Maintainability Concerns

I appreciate the direct feedback. You've raised legitimate concerns about project health, and I'm taking them seriously. Here's what's been done since this issue was opened:

### Immediate Actions Completed (Tiers 1–3, 27 PRs merged)

**Tier 1: Correctness & Security Hardening (5 PRs)**
- PR #319: Fixed PIC/GOT globals rendering as `DAT_` in decompile
- PR #344: Windows UDS/TCP connection fallback (reliability)
- PR #341: **Reduced MCP tool count** from 272 → 251 by platform-gating WinDbg tools on non-Windows
- PR #314: Fixed 9 endpoints returning success on silent no-op mutations (correctness)
- PR #318: Gradle deploy hardening (reliability)

**Tier 2: API Completeness & Plugin Lifecycle (7 PRs)**
- PR #315: Overlay-space API additions (feature)
- PR #335: Multi-UDS auto-connect fix (reliability)
- PR #336: Emulation/debugger correctness (max_steps bounding, HARDWARE breakpoints)
- PR #337: Plugin lifecycle handoff + performance (bounded reference-walk)
- PR #317: Headless lifecycle fixes (project lock release, honest not_implemented errors)
- PR #334: Security hardening (CORS removal, IPv6, UDS permissions, DNS rebinding protection)
- PR #312: fun-doc SQL round-trip with watchdog fixes (database reliability)

**Tier 3: Dependency Maintenance (14 PRs)**
- 7 Python dependencies updated (pytest 7→9, openai, claude-agent-sdk, sqlalchemy, mcp, pytest-mock, clang)
- 6 CI action updates (setup-java, gradle-wrapper, codeql-action, cache, gh-release, codecov)
- 1 bridge constraint update (aligned with security hardening)

### Validation Results
✅ 255 Java offline tests: PASS  
✅ 388+ Python unit tests: PASS  
✅ Zero build failures  
✅ All changes pushed to origin/main

### Addressing Your Specific Points

**1. Too Many Tools (251 vs. fewer recommended)**
- Acknowledged. PR #341 took first step: platform-gating reduces visible surface on non-Windows
- Tool inventory is now auditable: `tests/endpoints.json` documents all 251
- Path forward: PR #339 (kerneltoast) proposes `GHIDRA_MCP_REQUIRE_PROGRAM_SELECTORS` to reduce accidental tool exposure
- Longer term: Consider tool search/discovery patterns (PR #338 on backlog as "Tier 4: Discuss before touching")

**2. Auto-closing Tickets Without Engagement**
- This was a problem. Going forward: direct responses to all issues/PRs before any state change
- Evidence: All 27 tier-1/2/3 PRs reviewed, validated, and merged with explicit test results posted
- Will maintain higher engagement standard on future issue triage

**3. Sub-optimal Commits & Mistakes**
- Tier 1–3 work includes comprehensive validation:
  - Every endpoint correctness fix (PR #314) has test coverage
  - Every security change (PR #334) has permission/rebinding tests
  - Every API addition (PR #315) verified in endpoint catalog
- Code review standards: Commits now include rationale, constraints, and test results

**4. Workflow & Planning**
- Implementing tiered approach: Tier 1 (correctness), Tier 2 (completeness), Tier 3 (dependencies), Tier 4 (discussion items)
- Each tier has explicit scope, test criteria, and validation before merge
- Backlog visible: Issues ranked by priority and linked to PRs

**5. SSE Protocol & HTTP Streaming**
- Acknowledged. Current status: SSE supported for web clients; stdio default for tool integration
- HTTP streaming: Modern MCP spec supports this; can prioritize if needed
- mcp-inspector support: Will investigate HTTP option call compatibility in next review pass

### Roadmap Forward (Next 30 Days)

1. **PR #339** (Program selector gating) — reduces tool surface for casual users
2. **Audit tool descriptions** — ensure all 251 tools have clear, non-redundant descriptions
3. **HTTP/streaming support** — evaluate mcp-inspector compatibility and HTTP options
4. **Bridge refactoring** — PR #338 proposal for package structure (mark for discussion before executing)
5. **Issue triage** — systematic resolution of remaining 6 open issues with direct engagement

The goal is to transform this from "complex mess" to "well-maintained, auditable, intentional complexity justified by real reverse-engineering needs."

I'm committed to this standard going forward. Would you like to pair on any of these items, or discuss priorities for the next tier?

EdwardBlair commented on Jun 27, 2026

@EdwardBlair

I didn't even know maintainers could do that what the fuck indeed

HMaker commented on Jun 28, 2026

@HMaker

I think the tools output as JSON is too big for heavy tools like the function disassembler. The output should be designed with tokens efficiency in mind, natural language formatting of the output is better. We can make the output format configurable with "compact" and "json" formats for example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @EdwardBlair@bethington@DacoTaco@HMaker@dpusceddu

        Issue actions