Control X from the command line, and find things on the screen

 Control X from the command line for scripts, and do "visual scraping" to
 find things on the screen. The control interface allows mouse movement,
 clicking, button up/down, key up/down, etc, using the XTest extension.
 The visgrep program find images inside of images and reports the
 coordinates, allowing programs to find buttons, etc, on the screen to click
 on.
 .
 If you just want to control X, you may find xdotool easier to use.