-
Notifications
You must be signed in to change notification settings - Fork 15
feat(source): implement advanced search engine source #50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
wanghaoxue0
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good from my side
quantmind/models/search.py
Outdated
| snippet: str | ||
| source: str = "search" | ||
| query: Optional[str] = None | ||
| meta_info: Dict[str, Any] = {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the function of meta_info?
quantmind/sources/search_source.py
Outdated
| the value in the config. | ||
| site: Restrict search to a specific domain. | ||
| filetype: Search for specific file types. | ||
| start_date: Start date for search results (YYYY-MM-DD). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the datetime format is wrong?
Add something like this?
@field_validator("start_date", "end_date")
@classmethod
def _validate_date(cls, v):
if v is None:
return v
if not re.match(r"^\d{4}-\d{2}-\d{2}$", v):
raise ValueError("date must be YYYY-MM-DD")
return v
b8179f7 to
2c3ce59
Compare
Description
This pull request introduces a new SearchSource that enables fetching and processing content from the DuckDuckGo search engine. This new source is designed to be a robust and flexible way to incorporate web search results into the QuantMind framework. This PR complete part of the demand in #35
The key features of this implementation are:
Checklist