Limit search to path instead of domain?

[1kastner]

Could it be possible to restrict the search to a certain path?
A bad example would be to restrict a search to http://google.com/maps/ and ignore results which are in other "subdirectories" of http://google.com/.
Using "domain" for this purpose does not work.

[c4software]

Hi,

Sorry for the delay. You can do it via

--exclude "maps/"

But it has to be exhaustive.

You wan't something generic for all subfolders?

[1kastner]

Well, actually it is an include logic which is not yet implemented in https://github.com/c4software/python-sitemap/blob/master/main.py

[davidcx89]

I agree that it would be cool to have an "include" function in the crawler.
1kastner, I think your phrase "A bad example" may have read the opposite way to crsoftware.

[1kastner]

@davidcx89 yeap, sorry for bad phrasing, I maybe should have put more effort on describing the issue.

If I'll find the time there might be a pull request somewhen soon.

[c4software]

Hi,

An include pattern is indeed a great idea. Something with reggex would be really great.

I will try to doing this quickly. Maybe this weekend.