WASP send-click challenges

Oct 28, 2013 at 3:42 PM
I am having challenges with WASP send-click. The demo script below shows the challenges. Firefox will accept the click on the File menu, but not the click on the main web page display. IE & Chrome ignore the clicks. Notepad ignores the click on the File menu, but does correctly click on the main edit area, select all text and copy it to the window clipboard. Any and all suggestions would be appreciated.
# Initialization
Import-Module c:\wasp\wasp.dll
Add-Type -Assembly System.Windows.Forms 
clear-host
$str = "the quick brown fox`njumped over the lazy dog`n"

# Set up apps to test.
$app_list = $MyInvocation.UnboundArguments
if (!$app_list) {
   $app_list = "firefox", "iexplore", "chrome", "notepad"
}

# Main loop
foreach ($app in $app_list) {

   # Set coordinates to click on for the app.
   switch ($app) {
      chrome {
         $x1 = 20 # Apps button location
         $y1 = 80
         $x2 = 100 # main web page display
         $y2 = 150
      }
      iexplore {
         $x1 = 20 # File menu location
         $y1 = 80
         $x2 = 100
         $y2 = 200
      }
      firefox {
         $x1 = 20 # File menu location
         $y1 = 35
         $x2 = 100 # main web page display
         $y2 = 150
      }
      notepad {
         $x1 = 20 # File menu
         $y1 = 35
         $x2 = 40 # 3rd line of main editing window
         $y2 = 100
         # send-keys -window $h -keys "$str $str $str $str"
      }

      default {
         "ERROR: unknown app=$app"
         continue
      }
   }
   "app=$app x1=$x1 y1=$y1 x2=$x2 y2=$y2"

   # Start the application, get the window object, save object parms.
   start-process -filepath "$app"
   start-sleep -s 5
   if ($app -eq "iexplore") {
      $w = select-window -title *explorer*
   } else {
      $w = select-window -title *$app*
   }
   $w = $w | select-object -first 1
   $h = $w.handle
   $p = $w.processid
   $t = $w.title
   "app=$app h=$h p=$p t=$t"

   # If no window handle found, complain and move on.
   if (!$h) {
      "ERROR: app=$app got null handle"
      continue
   }

   # Get window position and mouse offsets.
   $pos=Get-WindowPosition $h
   $l = $pos.Location
   $x_off = $l.X
   $y_off = $l.Y
   "pos=$pos offset x=$x_off y=$y_off"

   # Move mouse, then click on first location.
   [System.Windows.Forms.Cursor]::Position = New-Object System.Drawing.Point(($x_off + $x1), ($y_off +$y1))
   start-sleep -s 1
   send-click -left $x1 -top $y1 -window $h -button left

   # Wait, then close first menu
   start-sleep -s 3
   send-keys -window $h -keys "{Esc}"

   # Add text for notepad window
   if ($app -eq "notepad") {
      send-keys -window $h -keys "${str}${str}${str}${str}${str}${str}${str}${str}"
   }

   # Move mouse, then click on second location.
   start-sleep -s 1
   [System.Windows.Forms.Cursor]::Position = New-Object System.Drawing.Point(($x_off + $x2), ($y_off +$y2))
   send-click -window $h -left $x2 -top $y2 -button left

   # Try to select all text & copy it.
   start-sleep -s 1
   send-keys -window $h -keys "^a"
   start-sleep -s 1
   send-keys -window $h -keys "^c"

   # Get rid of app & window.
   "stopping $app"
   start-sleep -s 1
   stop-process -id $p -force
   start-sleep -s 5 # wait for stop to take effect
}
Coordinator
Oct 28, 2013 at 8:28 PM
Short version (let me know if you need more, but I'm on my way out of the office):

You should try not to automate base on mouse clicks unless you can't avoid it -- mouse clicks automation is the most fragile: it breaks when resolution changes, or font-sizes change, and whenever things are moved on a UI even by just a few pixels. Use keystrokes (Alt+* hotkeys) for menus and buttons whenever possible. This is something that the new module is much better about, since it gives you Invoke-* cmdlets for stuff like buttons and menus...

Additionally, you shouldn't really drive browser automation with generic automation tools -- you need something that gets into the page and lets you automate based on the rendered DOM. My choice for browser automation is Selenium. I've done some work with Selenium in PowerShell using the WebDriver apis (I wrote a blog post and posted something to PoshCode, I think) and there's a project here on CodePlex (SePx? SePs? something like that) which claims to be a full port for PowerShell (I haven't tried that).
Oct 30, 2013 at 12:58 PM
Thanks for the info on upcoming WASP2. I fully agree that using X,Y mouse coordinates is a very fragile automation approach, but sometimes its the only way. I had that experience several years back using the Network Automation tool set. I am trying to understand what I can and cant do with WASP. I will make use of hot keys as much as possible, and will check out the other tools you mentioned. Thanks again!