Unionizing categories' predicates

[yrashk]

I have a case where I am trying to use categories to supplement predicate clauses.

Let's say we have an object obj that imports preds category which implements pred/0. Calling obj::pred will work fine. Then, if we want to add clauses to it in obj, simply defining pred/0 will override preds's version of pred/0 and we need to explicitly fall back onto it by adding a clause pred :- ^^pred. Not pretty, but ok.

However, It gets really hairy if preds is a category that extends multiple categories providing their clauses for pred/0. Using ^^ won't work as it won't call all clauses but only ones found first when going through the list of extended categories.

Ultimately, I am trying to use categories to mix in different clauses but it seems that either they are not fit for purpose or I need to reconsider my approach altogether.

Any suggestions on how to best achieve this?

[pmoura]

Would the following approach work in your case:

:- protocol(p).

	:- public(p/1).

:- end_protocol.


:- category(cat1,
	implements(p)).

	p(1).

:- end_category.


:- category(cat2,
	implements(p)).

	p(2).

:- end_category.


:- category(cat3,
	implements(p)).

	p(3).

:- end_category.


:- category(cat,
	extends((cat1,cat2,cat3))).

	:- alias(cat1, [p/1 as p1/1]).
	:- alias(cat2, [p/1 as p2/1]).
	:- alias(cat3, [p/1 as p3/1]).

	p(X) :- ^^p1(X).
	p(X) :- ^^p2(X).
	p(X) :- ^^p3(X).
	
:- end_category.


:- object(obj,
	imports(cat)).

	p(0).
	p(X) :- ^^p(X).

:- end_object.

Assuming the code above is saved in a cats.lgt file:

?- {cats}.
true.

?- obj::p(X).
X = 0 ;
X = 1 ;
X = 2 ;
X = 3.

[yrashk]

Thank you for such a prompt response! Technically, it would indeed.

The only problem I see with it is that it is error-prone (need to always remember to do proper dispatching). Perhaps this can be solved with a macro to eliminate the need to do aliasing+dispatching manually? Any other approaches to consider?

P.S. The reason I want to eliminate the need to do it manually is that I intend to have a fairly big tree of [extended] categories, and at the scale, I expect mistakes to be made (at the very least, simply forgetting to include one of the extended categories)

[yrashk]

In the ideal world (within the constraints of my current Logtalk exposure, which is minimal), I'd love to declare some predicates cumulative (so that they are not overridden when extended). If I can do this in a protocol and/or category, then I'd have the above functionality easily implementable.

:- protocol(proto).

:- public([p/1]).

:- inheritable_clauses([p/1]). % Or whatever other, more appropriate, name.

:- end_protocol.

[pmoura]

That may be possible using the term-expansion mechanism:

https://logtalk.org/manuals/userman/expansion.html

You could use to generate at compile time the alias/2 directives in the solution I posted.

[yrashk]

Yeah. This is what I was reading through. Just need to figure out how to get the being-defined object or concept so I can find all the relevant categories it imports.

[pmoura]

See the Handbook section on Reflection. Also the documentation of the logtalk_load_context/2 predicate.

[yrashk]

Following up on this aspect.

I wonder if there's any specific reason why logtalk_load_context can't provide entity parameters (such as implements, extends, imports, etc. as they were given at the entity opening directive)?

It already provides the type of the entity, its identifier and so on. If it was able to provide the aforementioned information, then one wouldn't need to expand the opening directives themselves but simply retrieve it from what logtalk_load_context can give.

[yrashk]

It appears that this is rather trivial to implement:

logtalk_load_context(imports_category, Category) :-
   '$lgt_pp_imported_category_'(Category, _, _, _, _, _).

(and similar to other relations)

Or one can findall relations if that's desirable.

If this interests you, I can prepare a patch and send a PR.

Update: here's the PR: #174

[pmoura]

Agree that it's a good idea. We can also simplify the key names (e.g. imports instead of imports_category). I will look later into your pull request and the required changes. Thanks.

[yrashk]

I did that naming to be able to distinguish different relations (extends_protocol vs extends_category), etc.

[pmoura]

In a first glance, I was thinking that using the same names as in the opening entity directives would be nice. But that would require ensuring first what kind of entity we're in the processing of compiling. But there's two other more important issues: (1) when compiling parametric objects, these new keys need to access both entity and related entity so that any common variables are easily accessed; plus (2) we may want to also know the scope of the relation.

[yrashk]

(Please let me know if continuing to post my findings is inappropriate here.)

I think I may have found an approach that is not perfect, but has some benefits and is (arguably) less error-prone (though still a bit wordy).

Firstly, I introduced the requirement_inference category:

:- category(requirement_inference).

:- multifile(inferred_requirement/1).
:- public(inferred_requirement/1).

:- public(requirement/1).

requirement(R) :- ::requires(R).
requirement(R) :- inferred_requirement(R).

:- public(includes_category/2).

includes_category(Entity, Category) :-
  imports_category(Entity, Category).

includes_category(Entity, Category) :-
  imports_category(Entity, Category1),
  includes_category(Category1, Category).

includes_category(Category1, Category) :-
  extends_category(Category1, Category).

includes_category(Category1, Category) :-
  extends_category(Category1, Category2),
  includes_category(Category2, Category).

:- end_category.

At its core, it relies on multifile inferred_requirement/1 definition. This allows us to write the following:

:- category(c_compiler_requirement_inference, extends(requirement_inference)).

:- multifile(requirement_inference::inferred_requirement/1).

requirement_inference::inferred_requirement(c_compiler(_)) :-
  self(Self),
  Self::includes_category(Self, c_compiler_requirement_inference),
  c_files_present(Self). % fake implementation here for brevity

:- end_category.

It's not as succinct as I would like it to be because of

1. the need to repeat multifile
2. the use of includes_category/2 (see below) to filter out the applications of this inference when it is not directly or indirectly included

But it's not too bad, I suppose. And it doesn't require term expansion.

Now, returning to requirement_inference, the great things about it are:

1. there's a separation of what package requires explicitly (requires/1), what's inferred (inferred_requirement/1) and any
   requirement (requirement/1).
2. it's available through package (Package::requirement/1, etc.) when it is included through any actual inferences as they extend requirement_inference.

This concept also defines the includes_category/2 convenience helper for the inference categories to filter applications that are not applicable. The main problem with it is it easy to forget it, and then the inference will apply to all packages whether they included it or not.

I am also not sure my implementation of includes_category/2 is any good but it seems to work.

[pmoura]

Have you looked into the packs tool? There, a pack object simply implements a pack manifest protocol. Similar for pack registries. A pack is, I think, similar to your concept of package. Thus, a package would be represented by an object that simply implements a package protocol. Then, any actions over packages would query the package objects using that protocol. For example, to verify if a package requirements are fulfilled, a forall/2 loop would verify, for each requires/1 clause, if that particular requirement is satisfied. The argument of the requires/1 predicate could be e.g. interpreted as the name of the object that verifies some set of requirements. This object would implement a common protocol whose predicates would take as argument the name of the package object. It would be these requirements objects inheriting from those categories that provide a functionally cohesive (and thus composable) view of a set of requirements and their inferencing code.

[yrashk]

At the very basic level, these are indeed similar. However, in my case, I have a much more sophisticated system where packages import sets of inferences they want to compose as I am building a package management system for an ecosystem that has no single set of rules and is very loosely defined (if at all) requirements and their satisfaction properties. Thus I've devised a system of different inferences that take what's available, depending on the package, and produce their inferences (such as requirements, metadata, etc.). In some cases, these inferences are good to override (like fixing a license or description), and it's better to make them additive in some cases.

I wish I could just show some of it so it can get clearer.

[yrashk]

Ok, I've devised yet another term-expansion-based approach that I like more. It dramatically simplifies the code at the expense of slower runtime performance and less-than-obvious code execution paths.

term_expansion((:- inherit(F/A)),
         (T :-
           self(Self),
           extended::includes_category(Self, C),
           category_property(C, defines(F/A, _)),
           setup_call_cleanup(create_object(O, [extends(Self), imports(C)], [], []),
                              O::T,
                              abolish_object(O)))) :-
   list::length(L, A), T =.. [F|L].

Now, packages willing to augment a predicate instead of overriding it can use :- inherit(F/A).

Besides the aforementioned runtime issues, the above implementation still needs some quality-of-life improvements (like supporting a list of predicates, making the predicate discontiguous, etc.).

I like how this is explicit and simple to use, and included categories don't have to deal with any complexity of this inclusion scheme.

[pmoura]

Maybe there's a better name than inherit/1 for the directive to be expanded?

[yrashk]

I also don't love inherit name, but I haven't found a better one just yet.

[pmoura]

Thinking that the "many worlds" design pattern may be a good fit here, with a package being a world. See e.g.

https://logtalk.org/2019/11/13/many-worlds-design-pattern.html

See also the design_patterns example.

[yrashk]

Thank you so much for this reference!

[pmoura]

At the very basic level, these are indeed similar. However, in my case, I have a much more sophisticated system where packages import sets of inferences they want to compose as I am building a package management system for an ecosystem that has no single set of rules and is very loosely defined (if at all) requirements and their satisfaction properties.

It's not clear to me why a package need to import anything. Maybe I'm stuck in the view of packages as essentially declarative manifests, possibly implementing different protocols (with likely some common minimal protocol). It would be the details of the package manifest that would result in different sets of inferences being applied. Composable inferences would still apply but the packages would work as blueprints directing requirements satisfaction to different objects importing different sets of categories, with each category implementing some cohesive subset of rules.