ChatGPT plugin discovery
gFit Home Workouts: recommendation experiments
We tested direct, indirect, equipment and same-wording fitness prompts to see whether ChatGPT searches for plugins and recommends gFit.
ChatGPT plugin discovery
We tested direct, indirect, equipment and same-wording fitness prompts to see whether ChatGPT searches for plugins and recommends gFit.
Experiment 01
Hypothesis: prompts that ask for a workout program make ChatGPT call search_plugins.
100
prompts with direct intent
search_plugins
0
0%
web.run
10
10%
Result: ChatGPT answered every direct prompt itself. It never looked for a plugin.
Experiment 02
Hypothesis: how many fitness prompts can trigger search_plugins at all, so that gFit can be recommended.
1,937
runs
search_plugins
6
0.3%
suggest_plugins only
2
0.1%
web.run
701
36%
Result: only 6 runs out of 1,937 reached plugin search. We reviewed them on the next slide.
8 runs called search_plugins or suggest_plugins. Only 2 were organic.
4 prompts
Not organic: real users rarely ask for plugins.
2 prompts
suggest_plugins → Google Calendar, no plugin search.
2 prompts
The equipment prompt became the lead for the next tests.
Experiment 03
Hypothesis: fitness equipment is a topic that reliably triggers search_plugins.
100
equipment prompts
search_plugins
0
0%
web.run
62
62%
Result: equipment prompts send ChatGPT to the web for prices and brands, not to plugins.
Experiment 04
Hypothesis: the wording itself triggers search_plugins, so we reused it across 50 product categories.
150
runs: 50 prompts × 3 accounts
search_plugins
0
0%
web.run
51
34%
functions.exec)
2
1%
Result: the original equipment prompt didn't repeat (0 of 3). The earlier hit was random.
| Test | Runs | search_plugins | web.run | Knowledge |
|---|---|---|---|---|
| Direct prompts | 100 | 0 | 10 | 90 |
| Indirect prompts | 1,937 | 6 | 701 | 1,188 |
| Fitness equipment | 100 | 0 | 62 | 38 |
| "How do I choose…" | 150 | 0 | 51 | 97 |
| Total | 2,287 | 60.26% | 82436.0% | 1,41361.8% |