Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions crawl4ai/async_configs.py
Original file line number Diff line number Diff line change
Expand Up @@ -872,6 +872,7 @@ class BrowserConfig:
user_agent_generator_config (dict or None): Configuration for user agent generation if user_agent_mode is set.
Default: None.
text_mode (bool): If True, disables images and other rich content for potentially faster load times.
JavaScript still runs; set java_script_enabled=False to turn it off.
Default: False.
light_mode (bool): Disables certain background features for performance gains. Default: False.
extra_args (list): Additional command-line arguments passed to the browser.
Expand Down
2 changes: 0 additions & 2 deletions crawl4ai/browser_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ def build_browser_flags(config: BrowserConfig) -> List[str]:
"--blink-settings=imagesEnabled=false",
"--disable-remote-fonts",
"--disable-images",
"--disable-javascript",
"--disable-software-rasterizer",
"--disable-dev-shm-usage",
])
Expand Down Expand Up @@ -1129,7 +1128,6 @@ def _build_browser_args(self) -> dict:
"--blink-settings=imagesEnabled=false",
"--disable-remote-fonts",
"--disable-images",
"--disable-javascript",
"--disable-software-rasterizer",
"--disable-dev-shm-usage",
]
Expand Down
2 changes: 1 addition & 1 deletion docs/md_v2/complete-sdk-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -1443,7 +1443,7 @@ class BrowserConfig:
- Custom User-Agent string. If `None`, a default is used.
- You can also set `user_agent_mode="random"` for randomization (if you want to fight bot detection).
9. **`text_mode`** & **`light_mode`**:
- `text_mode=True` disables images, possibly speeding up text-only crawls.
- `text_mode=True` disables images, possibly speeding up text-only crawls. JavaScript still runs; set `java_script_enabled=False` to turn it off.
- `light_mode=True` turns off certain background features for performance.
10. **`avoid_ads`** & **`avoid_css`**:
- `avoid_ads=True` blocks requests to common ad and tracker domains (Google Analytics, DoubleClick, Facebook, Hotjar, etc.) at the browser context level. Reduces network overhead and memory usage.
Expand Down
2 changes: 1 addition & 1 deletion docs/md_v2/core/browser-crawler-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ class BrowserConfig:
- `user_agent_mode`: Set to `"random"` for randomization (helps fight bot detection).

12.⠀**`text_mode`** & **`light_mode`**
- `text_mode=True` disables images, possibly speeding up text-only crawls.
- `text_mode=True` disables images, possibly speeding up text-only crawls. JavaScript still runs; set `java_script_enabled=False` to turn it off.
- `light_mode=True` turns off certain background features for performance.

13.⠀**`avoid_ads`** & **`avoid_css`**
Expand Down
30 changes: 30 additions & 0 deletions tests/unit/test_text_mode_flags.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""text_mode must not pass --disable-javascript to Chromium.

Chromium has no --disable-javascript switch and ignores it, so text_mode never
disabled scripting; the entry only suggested it did. Turning JavaScript off is
BrowserConfig(java_script_enabled=False), which sets it on the browser context.

These are flag-construction checks only; no browser or network required.
"""

from crawl4ai import BrowserConfig
from crawl4ai.browser_manager import BrowserManager, ManagedBrowser


def _text_mode_flag_sets():
config = BrowserConfig(headless=True, text_mode=True)
return (
ManagedBrowser.build_browser_flags(config),
BrowserManager(browser_config=config)._build_browser_args()["args"],
)


def test_text_mode_omits_the_dead_disable_javascript_switch():
for flags in _text_mode_flag_sets():
assert "--disable-javascript" not in flags


def test_text_mode_keeps_the_switches_that_take_effect():
for flags in _text_mode_flag_sets():
assert "--blink-settings=imagesEnabled=false" in flags
assert "--disable-remote-fonts" in flags