In this Blog, I will show you how I built a macro recorder in Python to record all mouse and keyboard operations such as clicks, scrolling, hold&drag, key presses etc.
In this blog post, we'll explore a Python script that leverages libraries such as pynput
, pyautogui
, keyboard
, and others to create a versatile recorder and replayer for mouse and keyboard events.
GIthub: https://github.com/biohacker0/maCrow
PermalinkIntroduction
The script provides a command-line interface to either record user actions in real-time or replay a previously recorded sequence. The recording captures mouse movements, clicks, scrolls, and keyboard events, saving them to a JSON file. The replay function reads the recorded actions from the file and simulates the user's interactions.
Sure, here is a table that you can copy and paste on Hashnode:
Feature | Description |
Mouse Recording | Captures mouse movements, clicks, and scrolls in real-time. |
Keyboard Recording | Captures keyboard key presses and releases, specifying the event type (down or up). |
Countdown Animation | Utilizes tkinter to provide a visual countdown animation during the initiation of recording or replay. |
Flexible File Handling | Allows users to specify a file name for saving recorded actions; default filename is “mouse_keyboard_actions.json.” |
Mouse Replay | Accurately replays recorded mouse movements, clicks, and releases. Supports double-click detection. |
Keyboard Replay | Replays recorded keyboard key presses and releases, introducing a customizable delay between events. |
User-Friendly Interface | Utilizes argparse for a command-line interface, providing clear instructions for users. |
Cross-Boundary Mouse Handling | Ensures that the mouse stays within the screen boundaries during recording. |
Transparent Window | Creates a transparent tkinter window for a visually appealing countdown animation. |
Error Handling | Gracefully handles interruptions (e.g., KeyboardInterrupt) during recording. |
To copy this table, select the entire table and press Ctrl+C
on Windows or Cmd+C
on Mac. Then, navigate to your Hashnode blog post and paste the table using Ctrl+V
on Windows or
PermalinkDependencies
The script relies on several Python libraries:
pynput
: Handles mouse and keyboard events.pyautogui
: Controls the mouse position and button presses.keyboard
: Manages keyboard events.mouse
: Retrieves the current mouse position.
PermalinkGetting Started
Before diving into the details of the code, make sure to install the required dependencies using the following:
pip install pynput pyautogui keyboard mouse
PermalinkCode Overview
PermalinkScreen Resolution
The script starts by obtaining the screen resolution using pyautogui.size()
. This information is crucial for maintaining mouse positions within the screen boundaries.
codescreen_width, screen_height = pyautogui.size()
PermalinkCountdown Animation
A countdown animation is displayed on the screen using tkinter
before recording or replaying starts. The countdown_animation
function creates a transparent full-screen window with a countdown label.
countdown_animation(root):
# ...
PermalinkRecording
The record
function initializes listeners for mouse and keyboard events. It captures actions such as mouse movements, clicks, scrolls, and keyboard inputs in real-time. The actions are stored in a list and later saved to a JSON file.
record(filename):
# ...
PermalinkReplay
The replay
function reads the recorded actions from a JSON file and replays them. It simulates mouse movements, clicks, scrolls, and keyboard inputs based on the recorded data.
replay(filename, key_delay=0.1):
# ...
PermalinkCommand Line Interface
The script uses the argparse
module to provide a user-friendly command-line interface. Users can choose between recording or replaying, and an optional --file
argument allows specifying the filename for saving or loading actions.
__name__ == '__main__':
parser = argparse.ArgumentParser(description='Mouse and Keyboard Recorder/Replayer')
parser.add_argument('command', choices=['record', 'replay'], help='Choose command: record or replay')
parser.add_argument('--file', default='mouse_keyboard_actions.json',
help='File to save mouse and keyboard actions (default: mouse_keyboard_actions.json)')
args = parser.parse_args()
if args.command == 'record':
record(args.file)
elif args.command == 'replay':
replay(args.file)
PermalinkUsage
To record user actions, run the script with the record
command:
python script.py record --file my_actions.json
To replay the recorded actions, use the replay
command:
python script.py replay --file my_actions.json
Permalink1. Countdown Animation and GUI (Tkinter):
Permalinkcountdown_animation(root)
:
Canvas and Label Widgets:
The
countdown_animation
function utilizes the Tkinter library to create a graphical user interface (GUI).A
Canvas
widget is employed to set up a blank space for drawing the countdown animation.A
Label
widget is used to display the countdown text.
Animation Display:
The
countdown_label
is dynamically updated to reflect the countdown status, whether it is recording or replaying.The
root.update()
method ensures that the GUI is refreshed in real-time.A short
time.sleep(1)
is introduced to create a delay between countdown updates, providing a visually appealing animation.
Cleanup:
- After the countdown animation completes, the
canvas.destroy()
method removes the Canvas widget.
- After the countdown animation completes, the
Permalinkcount_down_animation_config(mode)
:
Tkinter Window Configuration:
The
count_down_animation_config
function initializes the Tkinter window (root
) with specific attributes.The window is made transparent, topmost, and without decorations to create a clean and distraction-free display.
The
root.mode
attribute is set to indicate whether it's in recording or replay mode.
Visualization:
The window dimensions are set to match the screen resolution, ensuring the animation covers the entire screen.
The window is initially visible but is later hidden using
root.withdraw()
to provide a smooth transition to the main recording or replay process.
Permalink2. Recording (record
function):
PermalinkMouse and Keyboard Event Listeners:
Mouse Events (
pynput.mouse.Listener
):on_move(x, y)
: Captures mouse movement by appending position and time difference to theactions
list.on_click(x, y, button, pressed)
: Records mouse click events, distinguishing between press and release actions. Implements a simple double-click detection mechanism.on_scroll(x, y, dx, dy)
: Records mouse scroll events with position, scroll amount, and time difference.
Keyboard Events (
keyboard
Library):on_key_event(event)
: Records keyboard events, including key presses and releases, along with the event type and time difference.
PermalinkMain Loop:
Mouse Movement Simulation:
The script continuously updates the mouse position using the
mouse.get_position()
method.The
set_cursor_pos
function ensures the mouse stays within the screen boundaries.
Interrupt Handling:
The recording loop continues until a
KeyboardInterrupt
is raised (Ctrl + C).Upon interruption, the script stops the mouse and keyboard listeners and saves the recorded actions to a JSON file.
PermalinkSmoothing Mouse Movements:
- A brief sleep (
time.sleep(0.05)
) is introduced to control the loop speed, resulting in smoother mouse movement recordings.
Permalink3. Replaying (replay
function):
Permalinkreplay(filename, key_delay=0.1)
:
Reading Recorded Actions:
- The script reads the recorded actions from a JSON file using
json.load(file)
.
- The script reads the recorded actions from a JSON file using
PermalinkMain Loop:
Mouse and Keyboard Input Simulation:
The script iterates through the recorded actions, simulating mouse and keyboard input.
pyautogui
is used to simulate mouse movements and clicks.mouse
andkeyboard
libraries are employed for more granular control over mouse clicks and keyboard inputs.
PermalinkDouble Click Detection:
- A threshold (
double_click_threshold
) is used to identify and handle double-click events.
PermalinkDelay:
- A delay (
time.sleep(key_delay)
) is introduced between key presses during replay to control the speed of the replay.
Permalink4. Command Line Interface (CLI):
Argument Parsing (
argparse
):The script uses
argparse
for parsing command-line arguments.Accepts either
record
orreplay
as the main command.The
--file
argument allows users to specify the output (record) or input (replay) JSON file.
PermalinkIssues I faced
When I was using pynput and its listener to listen to key presses by keyboard, it would work for single key presses, but I struggled a lot with key combos like
ctrl+c
ctrl+backspace
.I had to write a formula where I was storing time when a key was pressed, and if multiple keys were pressed during a key press and its release in same time frame then I would store that as a combo to be executed as a hotkey.
Now combo like
ctrl+backspace
that were special keys would work but notctrl+a
orctrl+c
, a combo of special key and normal character key would give issues to me : in my Jsonctrl+c
would be recorded as /x1600 and in terminal, a black heart symbol was printed when pyautogui executed that.