fully working transcription
This commit is contained in:
@@ -7,7 +7,7 @@ Press a toggle key, speak, and get instant text input. Built natively for Waylan
|
||||
- **Toggle workflow**: Press once to start recording, press again to stop and inject text
|
||||
- **Wayland native**: Purpose-built for Wayland compositors - no legacy X11 dependencies or hacky workarounds
|
||||
- **Real-time feedback**: Desktop notifications for recording states and transcription status
|
||||
- **Multiple transcription backends**: OpenAI Whisper (planned: whisper.cpp for local processing)
|
||||
- **Multiple transcription backends**: OpenAI Whisper (planned: whisper.cpp for local processing, and more)
|
||||
- **Smart text injection**: Clipboard save/restore with direct typing fallback
|
||||
- **Daemon architecture**: Lightweight control plane with efficient pipeline management
|
||||
|
||||
@@ -72,7 +72,7 @@ sudo apt install pipewire-pulse pipewire-bin wl-clipboard
|
||||
sudo dnf install pipewire-utils wl-clipboard
|
||||
```
|
||||
|
||||
**For text injection (recommended):**
|
||||
**For text injection:**
|
||||
|
||||
```bash
|
||||
# Arch Linux
|
||||
@@ -196,7 +196,7 @@ Configuration will be read from `~/.config/hyprvoice/config.toml` (planned). Cur
|
||||
|
||||
Hyprvoice will support multiple transcription backends:
|
||||
|
||||
#### OpenAI Whisper API (Planned)
|
||||
#### OpenAI Whisper API
|
||||
|
||||
Fast, accurate cloud-based transcription:
|
||||
|
||||
@@ -219,6 +219,31 @@ model_path = "~/models/ggml-base.en.bin"
|
||||
threads = 4
|
||||
```
|
||||
|
||||
#### Text Injection (Current)
|
||||
|
||||
Configurable text injection with multiple modes:
|
||||
|
||||
```toml
|
||||
[injection]
|
||||
mode = "fallback" # "clipboard", "type", or "fallback"
|
||||
always_copy_clipboard = true
|
||||
restore_clipboard = true
|
||||
wtype_timeout = "5s"
|
||||
clipboard_timeout = "3s"
|
||||
```
|
||||
|
||||
**Injection Modes:**
|
||||
|
||||
- **`fallback`** (default): Try direct typing first, fallback to clipboard
|
||||
- **`type`**: Direct typing using wtype only
|
||||
- **`clipboard`**: Copy to clipboard only
|
||||
|
||||
**Behavior:**
|
||||
|
||||
- `always_copy_clipboard = true`: Always copy text to clipboard regardless of mode
|
||||
- `restore_clipboard = true`: Save and restore original clipboard content
|
||||
- Smart fallback ensures text injection always succeeds when possible
|
||||
|
||||
### Service Configuration
|
||||
|
||||
#### Systemd Service
|
||||
@@ -256,17 +281,17 @@ systemctl --user enable --now hyprvoice.service
|
||||
|
||||
## Development Status
|
||||
|
||||
| Component | Status | Notes |
|
||||
| --------------------- | ------ | --------------------------------- |
|
||||
| Core daemon & IPC | ✅ | Unix socket control plane |
|
||||
| Recording workflow | ✅ | Toggle recording via PipeWire |
|
||||
| Audio capture | ✅ | Efficient PipeWire integration |
|
||||
| Desktop notifications | ✅ | Status feedback via notify-send |
|
||||
| OpenAI transcription | ✅ | HTTP API integration |
|
||||
| Text injection | ⏳ | Clipboard + typing implementation |
|
||||
| Configuration system | ⏳ | TOML-based user settings |
|
||||
| Comprehensive tests | ⏳ | Pipeline and integration testing |
|
||||
| whisper.cpp support | ⏳ | Local model inference |
|
||||
| Component | Status | Notes |
|
||||
| --------------------- | ------ | -------------------------------- |
|
||||
| Core daemon & IPC | ✅ | Unix socket control plane |
|
||||
| Recording workflow | ✅ | Toggle recording via PipeWire |
|
||||
| Audio capture | ✅ | Efficient PipeWire integration |
|
||||
| Desktop notifications | ✅ | Status feedback via notify-send |
|
||||
| OpenAI transcription | ✅ | HTTP API integration |
|
||||
| Text injection | ✅ | Clipboard + wtype with fallback |
|
||||
| Configuration system | ⏳ | TOML-based user settings |
|
||||
| Comprehensive tests | ⏳ | Pipeline and integration testing |
|
||||
| whisper.cpp support | ⏳ | Local model inference |
|
||||
|
||||
**Legend**: ✅ Complete · ⏳ Planned
|
||||
|
||||
@@ -290,8 +315,8 @@ flowchart LR
|
||||
end
|
||||
subgraph Pipeline
|
||||
A["Audio Capture"]
|
||||
T["Transcribing (ASR TBD)"]
|
||||
I["Injecting (stub)"]
|
||||
T["Transcribing"]
|
||||
I["Injecting (wtype + clipboard)"]
|
||||
end
|
||||
N["notify-send/log"]
|
||||
|
||||
@@ -404,11 +429,34 @@ sudo apt install libnotify-bin # Ubuntu/Debian
|
||||
|
||||
#### Text Injection Issues
|
||||
|
||||
**Text not appearing (when implemented):**
|
||||
**Text not appearing:**
|
||||
|
||||
- Ensure cursor is in a text field when toggling off recording
|
||||
- Check that `wtype` or clipboard tools are installed
|
||||
- Verify window manager supports the text injection method used
|
||||
- Check that `wtype` and `wl-clipboard` tools are installed:
|
||||
|
||||
```bash
|
||||
# Test wtype directly
|
||||
wtype "test text"
|
||||
|
||||
# Test clipboard tools
|
||||
echo "test" | wl-copy
|
||||
wl-paste
|
||||
```
|
||||
|
||||
- Verify Wayland compositor supports text input protocols
|
||||
- Check injection mode in configuration (fallback mode is most robust)
|
||||
|
||||
**Clipboard issues:**
|
||||
|
||||
```bash
|
||||
# Install wl-clipboard if missing
|
||||
sudo pacman -S wl-clipboard # Arch
|
||||
sudo apt install wl-clipboard # Ubuntu/Debian
|
||||
|
||||
# Test clipboard functionality
|
||||
wl-copy "test text"
|
||||
wl-paste
|
||||
```
|
||||
|
||||
### Debug Mode
|
||||
|
||||
@@ -453,6 +501,7 @@ hyprvoice/
|
||||
├── internal/
|
||||
│ ├── bus/ # IPC (Unix socket) + PID management
|
||||
│ ├── daemon/ # Control daemon (lifecycle management)
|
||||
│ ├── injection/ # Text injection (clipboard + wtype)
|
||||
│ ├── notify/ # Desktop notification integration
|
||||
│ ├── pipeline/ # Audio processing pipeline + state machine
|
||||
│ ├── recording/ # PipeWire audio capture
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
package injection
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// getClipboard retrieves the current clipboard content using wl-paste
|
||||
func getClipboard(ctx context.Context, timeout time.Duration) (string, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
cmd := exec.CommandContext(ctx, "wl-paste", "--no-newline")
|
||||
output, err := cmd.Output()
|
||||
if err != nil {
|
||||
// wl-paste returns non-zero exit code if clipboard is empty or unavailable
|
||||
// This is normal behavior, so we return empty string instead of error
|
||||
return "", nil
|
||||
}
|
||||
|
||||
return string(output), nil
|
||||
}
|
||||
|
||||
// setClipboard sets the clipboard content using wl-copy
|
||||
func setClipboard(ctx context.Context, text string, timeout time.Duration) error {
|
||||
ctx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
cmd := exec.CommandContext(ctx, "wl-copy")
|
||||
cmd.Stdin = strings.NewReader(text)
|
||||
|
||||
if err := cmd.Run(); err != nil {
|
||||
return fmt.Errorf("wl-copy failed: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkClipboardAvailable checks if wl-clipboard tools are available
|
||||
func checkClipboardAvailable() error {
|
||||
if _, err := exec.LookPath("wl-copy"); err != nil {
|
||||
return fmt.Errorf("wl-copy not found: %w (install wl-clipboard)", err)
|
||||
}
|
||||
|
||||
if _, err := exec.LookPath("wl-paste"); err != nil {
|
||||
return fmt.Errorf("wl-paste not found: %w (install wl-clipboard)", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package injection
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Injector interface for text injection
|
||||
type Injector interface {
|
||||
Inject(ctx context.Context, text string) error
|
||||
}
|
||||
|
||||
// Config for text injection
|
||||
type Config struct {
|
||||
Mode string // "clipboard", "type", "fallback"
|
||||
AlwaysCopyClipboard bool // Always copy to clipboard regardless of mode
|
||||
RestoreClipboard bool // Restore original clipboard after injection
|
||||
WtypeTimeout time.Duration // Timeout for wtype commands
|
||||
ClipboardTimeout time.Duration // Timeout for clipboard operations
|
||||
}
|
||||
|
||||
// DefaultConfig returns sensible defaults for injection
|
||||
func DefaultConfig() Config {
|
||||
return Config{
|
||||
Mode: "fallback",
|
||||
AlwaysCopyClipboard: true,
|
||||
RestoreClipboard: true,
|
||||
WtypeTimeout: 5 * time.Second,
|
||||
ClipboardTimeout: 3 * time.Second,
|
||||
}
|
||||
}
|
||||
|
||||
// injector implements the Injector interface
|
||||
type injector struct {
|
||||
config Config
|
||||
}
|
||||
|
||||
// NewInjector creates a new injector with the given config
|
||||
func NewInjector(config Config) Injector {
|
||||
return &injector{
|
||||
config: config,
|
||||
}
|
||||
}
|
||||
|
||||
// NewDefaultInjector creates an injector with default configuration
|
||||
func NewDefaultInjector() Injector {
|
||||
return NewInjector(DefaultConfig())
|
||||
}
|
||||
|
||||
// Inject performs text injection based on the configured mode
|
||||
func (i *injector) Inject(ctx context.Context, text string) error {
|
||||
if text == "" {
|
||||
return fmt.Errorf("cannot inject empty text")
|
||||
}
|
||||
|
||||
// Always copy to clipboard if configured
|
||||
var originalClipboard string
|
||||
var err error
|
||||
|
||||
if i.config.AlwaysCopyClipboard || i.config.Mode == "clipboard" || i.config.Mode == "fallback" {
|
||||
if err := checkClipboardAvailable(); err != nil {
|
||||
return fmt.Errorf("clipboard tools not available: %w", err)
|
||||
}
|
||||
|
||||
if i.config.RestoreClipboard {
|
||||
originalClipboard, _ = getClipboard(ctx, i.config.ClipboardTimeout)
|
||||
}
|
||||
|
||||
if err := setClipboard(ctx, text, i.config.ClipboardTimeout); err != nil {
|
||||
return fmt.Errorf("failed to copy text to clipboard: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Handle different injection modes
|
||||
switch i.config.Mode {
|
||||
case "clipboard":
|
||||
// Already handled above
|
||||
return nil
|
||||
|
||||
case "type":
|
||||
err = typeText(ctx, text, i.config.WtypeTimeout)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to type text: %w", err)
|
||||
}
|
||||
|
||||
case "fallback":
|
||||
// Try typing first, fallback to clipboard
|
||||
err = typeText(ctx, text, i.config.WtypeTimeout)
|
||||
if err != nil {
|
||||
// Typing failed, but clipboard is already set from above
|
||||
// Just log the typing error but don't fail the injection
|
||||
return nil
|
||||
}
|
||||
|
||||
default:
|
||||
return fmt.Errorf("unsupported injection mode: %s", i.config.Mode)
|
||||
}
|
||||
|
||||
// Restore original clipboard if configured and we have it
|
||||
if i.config.RestoreClipboard && originalClipboard != "" {
|
||||
// Restore after a short delay to ensure the text has been processed
|
||||
go func() {
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
restoreCtx, cancel := context.WithTimeout(context.Background(), i.config.ClipboardTimeout)
|
||||
defer cancel()
|
||||
setClipboard(restoreCtx, originalClipboard, i.config.ClipboardTimeout)
|
||||
}()
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package injection
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"time"
|
||||
)
|
||||
|
||||
// typeText types the given text using wtype
|
||||
func typeText(ctx context.Context, text string, timeout time.Duration) error {
|
||||
ctx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
// Check if wtype is available
|
||||
if err := checkWtypeAvailable(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
cmd := exec.CommandContext(ctx, "wtype", text)
|
||||
|
||||
if err := cmd.Run(); err != nil {
|
||||
return fmt.Errorf("wtype failed: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkWtypeAvailable checks if wtype is available on the system
|
||||
func checkWtypeAvailable() error {
|
||||
if _, err := exec.LookPath("wtype"); err != nil {
|
||||
return fmt.Errorf("wtype not found: %w (install wtype package)", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/leonardotrapani/hyprvoice/internal/injection"
|
||||
"github.com/leonardotrapani/hyprvoice/internal/recording"
|
||||
"github.com/leonardotrapani/hyprvoice/internal/transcriber"
|
||||
)
|
||||
@@ -116,6 +117,19 @@ func (p *pipeline) run(ctx context.Context) {
|
||||
}
|
||||
}()
|
||||
|
||||
// Forward errors from component channels to unified pipeline error channel
|
||||
go func() {
|
||||
for err := range tErrCh {
|
||||
p.sendError("Transcription Error", "Transcription processing error", err)
|
||||
}
|
||||
}()
|
||||
|
||||
go func() {
|
||||
for err := range rErrCh {
|
||||
p.sendError("Recording Error", "Recording stream error", err)
|
||||
}
|
||||
}()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-frameCh:
|
||||
@@ -127,14 +141,6 @@ func (p *pipeline) run(ctx context.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
case err := <-tErrCh:
|
||||
p.handleTranscriberError(err)
|
||||
return
|
||||
|
||||
case err := <-rErrCh:
|
||||
p.handleRecordingError(err)
|
||||
return
|
||||
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
@@ -191,14 +197,6 @@ func (p *pipeline) sendError(title, message string, err error) {
|
||||
}
|
||||
}
|
||||
|
||||
func (p *pipeline) handleTranscriberError(err error) {
|
||||
p.sendError("Transcription Error", "Transcription processing error", err)
|
||||
}
|
||||
|
||||
func (p *pipeline) handleRecordingError(err error) {
|
||||
p.sendError("Recording Error", "Recording stream error", err)
|
||||
}
|
||||
|
||||
func (p *pipeline) handleInjectAction(ctx context.Context, recorder *recording.Recorder, t transcriber.Transcriber) {
|
||||
status := p.Status()
|
||||
|
||||
@@ -214,6 +212,7 @@ func (p *pipeline) handleInjectAction(ctx context.Context, recorder *recording.R
|
||||
|
||||
if err := t.Stop(ctx); err != nil {
|
||||
p.sendError("Transcription Error", "Failed to stop transcriber during injection", err)
|
||||
return
|
||||
}
|
||||
|
||||
transcriptionText, err := t.GetFinalTranscription()
|
||||
@@ -223,9 +222,16 @@ func (p *pipeline) handleInjectAction(ctx context.Context, recorder *recording.R
|
||||
}
|
||||
log.Printf("Pipeline: Final transcription text: %s", transcriptionText)
|
||||
|
||||
log.Printf("Pipeline: Simulating injection work")
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
log.Printf("Pipeline: Injection work done, returning to idle")
|
||||
injector := injection.NewDefaultInjector()
|
||||
|
||||
if err := injector.Inject(ctx, transcriptionText); err != nil {
|
||||
p.sendError("Injection Error", "Failed to inject text", err)
|
||||
} else {
|
||||
log.Printf("Pipeline: Text injection completed successfully")
|
||||
}
|
||||
|
||||
// Return to idle state after injection is complete
|
||||
p.setStatus(Idle)
|
||||
}
|
||||
|
||||
func (p *pipeline) Stop() {
|
||||
|
||||
Reference in New Issue
Block a user