In this Blog, I will show you how I built a macro recorder in Python to record all mouse and keyboard operations such as clicks, scrolling, hold&drag, key presses etc.

In this blog post, we'll explore a Python script that leverages libraries such as pynput, pyautogui, keyboard, and others to create a versatile recorder and replayer for mouse and keyboard events.

GIthub: https://github.com/biohacker0/maCrow

Introduction

The script provides a command-line interface to either record user actions in real-time or replay a previously recorded sequence. The recording captures mouse movements, clicks, scrolls, and keyboard events, saving them to a JSON file. The replay function reads the recorded actions from the file and simulates the user's interactions.

Sure, here is a table that you can copy and paste on Hashnode:

To copy this table, select the entire table and press Ctrl+C on Windows or Cmd+C on Mac. Then, navigate to your Hashnode blog post and paste the table using Ctrl+V on Windows or

Dependencies

The script relies on several Python libraries:

• pynput: Handles mouse and keyboard events.

• pyautogui: Controls the mouse position and button presses.

• keyboard: Manages keyboard events.

• mouse: Retrieves the current mouse position.

Getting Started

Before diving into the details of the code, make sure to install the required dependencies using the following:

pip install pynput pyautogui keyboard mouse

Code Overview

Screen Resolution

The script starts by obtaining the screen resolution using pyautogui.size(). This information is crucial for maintaining mouse positions within the screen boundaries.

 codescreen_width, screen_height = pyautogui.size()

Countdown Animation

A countdown animation is displayed on the screen using tkinter before recording or replaying starts. The countdown_animation function creates a transparent full-screen window with a countdown label.

 countdown_animation(root):
    # ...

Recording

The record function initializes listeners for mouse and keyboard events. It captures actions such as mouse movements, clicks, scrolls, and keyboard inputs in real-time. The actions are stored in a list and later saved to a JSON file.

record(filename):
    # ...

Replay

The replay function reads the recorded actions from a JSON file and replays them. It simulates mouse movements, clicks, scrolls, and keyboard inputs based on the recorded data.

replay(filename, key_delay=0.1):
    # ...

Command Line Interface

The script uses the argparse module to provide a user-friendly command-line interface. Users can choose between recording or replaying, and an optional --file argument allows specifying the filename for saving or loading actions.

__name__ == '__main__':
    parser = argparse.ArgumentParser(description='Mouse and Keyboard Recorder/Replayer')
    parser.add_argument('command', choices=['record', 'replay'], help='Choose command: record or replay')
    parser.add_argument('--file', default='mouse_keyboard_actions.json',
                        help='File to save mouse and keyboard actions (default: mouse_keyboard_actions.json)')
    args = parser.parse_args()

    if args.command == 'record':
        record(args.file)
    elif args.command == 'replay':
        replay(args.file)

Usage

To record user actions, run the script with the record command:

python script.py record --file my_actions.json

To replay the recorded actions, use the replay command:

python script.py replay --file my_actions.json

1. Countdown Animation and GUI (Tkinter):

countdown_animation(root):

• Canvas and Label Widgets:

  • The countdown_animation function utilizes the Tkinter library to create a graphical user interface (GUI).

  • A Canvas widget is employed to set up a blank space for drawing the countdown animation.

  • A Label widget is used to display the countdown text.

• Animation Display:

  • The countdown_label is dynamically updated to reflect the countdown status, whether it is recording or replaying.

  • The root.update() method ensures that the GUI is refreshed in real-time.

  • A short time.sleep(1) is introduced to create a delay between countdown updates, providing a visually appealing animation.

• Cleanup:

  • After the countdown animation completes, the canvas.destroy() method removes the Canvas widget.

count_down_animation_config(mode):

• Tkinter Window Configuration:

  • The count_down_animation_config function initializes the Tkinter window (root) with specific attributes.

  • The window is made transparent, topmost, and without decorations to create a clean and distraction-free display.

  • The root.mode attribute is set to indicate whether it's in recording or replay mode.

• Visualization:

  • The window dimensions are set to match the screen resolution, ensuring the animation covers the entire screen.

  • The window is initially visible but is later hidden using root.withdraw() to provide a smooth transition to the main recording or replay process.

2. Recording (record function):

Mouse and Keyboard Event Listeners:

• Mouse Events (pynput.mouse.Listener):

  • on_move(x, y): Captures mouse movement by appending position and time difference to the actions list.

  • on_click(x, y, button, pressed): Records mouse click events, distinguishing between press and release actions. Implements a simple double-click detection mechanism.

  • on_scroll(x, y, dx, dy): Records mouse scroll events with position, scroll amount, and time difference.

• Keyboard Events (keyboard Library):

  • on_key_event(event): Records keyboard events, including key presses and releases, along with the event type and time difference.

Main Loop:

• Mouse Movement Simulation:

  • The script continuously updates the mouse position using the mouse.get_position() method.

  • The set_cursor_pos function ensures the mouse stays within the screen boundaries.

• Interrupt Handling:

  • The recording loop continues until a KeyboardInterrupt is raised (Ctrl + C).

  • Upon interruption, the script stops the mouse and keyboard listeners and saves the recorded actions to a JSON file.

Smoothing Mouse Movements:

• A brief sleep (time.sleep(0.05)) is introduced to control the loop speed, resulting in smoother mouse movement recordings.

3. Replaying (replay function):

replay(filename, key_delay=0.1):

• Reading Recorded Actions:

  • The script reads the recorded actions from a JSON file using json.load(file).

Main Loop:

• Mouse and Keyboard Input Simulation:

  • The script iterates through the recorded actions, simulating mouse and keyboard input.

  • pyautogui is used to simulate mouse movements and clicks.

  • mouse and keyboard libraries are employed for more granular control over mouse clicks and keyboard inputs.

Double Click Detection:

• A threshold (double_click_threshold) is used to identify and handle double-click events.

Delay:

• A delay (time.sleep(key_delay)) is introduced between key presses during replay to control the speed of the replay.

4. Command Line Interface (CLI):

• Argument Parsing (argparse):

  • The script uses argparse for parsing command-line arguments.

  • Accepts either record or replay as the main command.

  • The --file argument allows users to specify the output (record) or input (replay) JSON file.

• 

Issues I faced

• When I was using pynput and its listener to listen to key presses by keyboard, it would work for single key presses, but I struggled a lot with key combos like ctrl+c ctrl+backspace .

• I had to write a formula where I was storing time when a key was pressed, and if multiple keys were pressed during a key press and its release in same time frame then I would store that as a combo to be executed as a hotkey.

• Now combo like ctrl+backspace that were special keys would work but not ctrl+a or ctrl+c , a combo of special key and normal character key would give issues to me : in my Json ctrl+c would be recorded as /x1600 and in terminal, a black heart symbol was printed when pyautogui executed that.