Add real transformer batch inference

[Mayukhga83]

Summary

• replace per-example transformer inference with padded whole-list batching and a single model forward pass
• preserve input order while handling uneven sequence lengths, token and span outputs, confidence filtering, taxonomy typing, and empty inputs
• reject mismatched prompt/answer lengths for both transformer and LLM batch paths
• keep benchmark memory reporting portable on platforms without the Unix resource module

Related issue

Closes #23.

Type of change

• Bug fix
• Feature
• Documentation
• Tests
• Refactor or maintenance

Testing

• ruff format --check lettucedetect/ lettucedetect_api/ tests/
• ruff check lettucedetect/ lettucedetect_api/ tests/ --extend-exclude lettucedetect/integrations/
• python -m pytest
• Other: ruff check scripts/benchmark_detectors.py

Checklist

• I kept the PR focused on one change.
• I added or updated tests/docs when needed.
• I checked that no secrets, API keys, or credentials are included.

Rights & sign-off (required)

• I certify that I have the right to submit this code and that it may be
  distributed under the repository's MIT license
  (see CONTRIBUTING).