24.01-scripting
Goal
Ability to create custom rules inside yaml configuration with fast execution.
Variants
Why not JavaScript, Lua, or WASM?
JavaScript and Lua are rich languages that require sandboxing to execute safely. Sandboxing is costly and factors into the “what will I let users evaluate?” question heavily when the answer is anything more than O(n) complexity.
CEL evaluates linearly with respect to the size of the expression and the input being evaluated when macros are disabled. The only functions beyond the built-ins that may be invoked are provided by the host environment. While extension functions may be more complex, this is a choice by the application embedding CEL.
But, why not WASM? WASM is an excellent choice for certain applications and is far superior to embedded JavaScript and Lua, but it does not have support for garbage collection and non-primitive object types require semi-expensive calls across modules. In most cases CEL will be faster and just as portable for its intended use case, though for node.js and web-based execution CEL too may offer a WASM evaluator with direct to WASM compilation.
Concrete use case
User creates if rules for certain errors.
custom-rules:
- message: "files inside dir/ can't be modified, only deletion allowed"
if: "diff.find(`^dir/.*$`).Additions > 0"
- message: "module `test_utils` can be imported only inside test files"
if: "diff.contains(`test_utils`, addition)"
How to make useful reporting
If without context
message: "files inside dir/ can't be modified, only deletion allowed"
if: "diff.find(`^dir/.*$`).Additions > 0"
- Easy to write
- BUT hard to find invalid commit or file
List return
message: "files inside dir/ can't be modified, only deletion allowed"
find: "testMap.filter(k, testMap[k].Integer > 100).map(k, testMap[k].StringField)"
error: files inside dir/ can't be modified, only deletion allowed
invalid objects: [ test-file-go, other-file.go ]
- Obvious which objects is invalid
- BUT additional
mapstep required - BUT it is not unified reporting format
Get error contexts from return value type
- No context (just message) - bool or unknown string return
- commit - filtered []Commit list return
- diff file - diff map keys string list or []FileDiff list return
- diff line (future feature) - like diff file but with
:11,28
message: "files inside dir/ can't be modified, only deletion allowed"
func: "testMap.filter(k, testMap[k].Integer > 100)"
- Obvious which objects is invalid
- Unified reporting format
cel-go problems
It possible to use native types by ext.NativeTypes(reflect.ValueOf(SomeStruct{}))
But struct fields in TitleCamelCase.
All macros in lower case, so it is good to have lower case fields names.
Options:
- Create patch to cel-go with customizable field reflection
- Copy and edit
ext.NativeTypes - Use protobuf generated structure
- Use type provider wrapper to rename field names
A lot of experiments with CEL were made.
Custom TypeProvider.FindStructFieldType is the best solution. It is simple and allows to use just Go structs and any case for field names (camel, snake and kebab)
Custom values can be retrieved like this.
&types.FieldType{
Type: types.StringType,
IsSet: func (target any) bool {
return true
},
GetFrom: func (target any) (any, error) {
return target.(SomeStruct).Map[fieldName], nil
},
}