Select vs Click
Based on my test - Never use the word "select" in Laya prompts—use "click" or "choose":
Prompts containing the English verb "Select [X]" cause Laya to predict the HTML "select" operation instead of clicking buttons.
Example: Write "Click the 2:00 PM time slot" instead of "Select the 2:00 PM time slot". This single tweak immediately eliminates ~15% of Laya's false failures.
Thanks for testing this! I reproduced it, but the cause is broader than the wording. When the page has any dropdown, the model over-predicts SELECT whether the goal says "select", "click", "choose" or "book". On a booking page with a time-zone dropdown, all five phrasings chose the dropdown (p≈0.85–0.97). Without the dropdown, all five clicked "2:00 PM". On held-out pages with a dropdown where the correct action is a click, 23% are predicted as SELECT (click accuracy 68% vs 81% on pages without one). Most of it comes from Mind2Web-style form pages.
Rewording the goal won't reliably help. What does: if the dropdown isn't part of the task, leave kind: "select" entries out of page["actions"], or check whether the chosen select option actually appears in your goal before executing it.
A proper fix needs contrastive training data (dropdown present, gold = click); I'm not retraining this model right now, so I'll leave this open as a known limitation.
understood - thanks for explaining.