Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cost-model): introduce MemoExt trait for property info #38

Merged
merged 10 commits into from
Nov 18, 2024

Conversation

xx01cyx
Copy link
Member

@xx01cyx xx01cyx commented Nov 18, 2024

To compute the cost for an expression, we need information about the schema & attribute ref (including attribute correlations). In the current optd, this is done by calling the optimizer core's methods. We were against this approach in previous discussions because we thought this makes the core and the cost model coupled too much -- we thereby eliminated the optimizer parameter and intended to get all these information from the storage/ORM.

However, there is a performance drawback of getting everything from ORM: the core should have all the information (schema & attribute ref) we need in memory -- it would be more efficient for them to be passed in by the core than querying the underlying external database. This also aligns more with the way the cascades optimizer works: building the memo table in a bottom-up approach and remembering everything.

Therefore, to avoid getting everything from ORM and still use one general interface for all types of node, we would need the core to implement a trait provided by the cost model, and the cost model will call the corresponding methods to get the information, i.e. MemoExt in this PR. This allows the core to remain ignorant of what the cost model needs for computing the cost.

@xx01cyx xx01cyx changed the title feat(cost-model): introduce trait MemoExt for property info feat(cost-model): introduce MemoExt trait for property info Nov 18, 2024
@xx01cyx xx01cyx requested review from skyzh and lanlou1554 November 18, 2024 02:28
Comment on lines 21 to 25
pub struct CostModelImpl<S: CostModelStorageLayer> {
storage_manager: CostModelStorageManager<S>,
default_catalog_source: CatalogSource,
_memo: Arc<dyn MemoExt>,
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether it's good to use Arc here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine, otherwise you'll need to add the memo type as part of the cost model impl's generic type parameters.

Co-authored-by: Lan Lou <62441979+lanlou1554@users.noreply.github.com>
@xx01cyx xx01cyx merged commit 9ba03e6 into main Nov 18, 2024
11 checks passed
@xx01cyx xx01cyx deleted the cost-model-memo-ext branch November 18, 2024 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants