Feat/filter control blocks #20
Conversation
* Feature/includeparameter (#14) * Add skeleton for inclusion list parameter Extend interface/skeleton for include_list * Input POinter Attempt to add input pointer parameter for control blocks * Definitions outside of if statements * Attempt to map the intended inclusion list * Try Virtual Function to keep Control Block in Explorer * No longer have get return * Attempt to add the meat of the function -Add methods and members to control blocks to get name of control block, specific control blocks within it, and all control blocks within it -add functions in control block explorer to parse inclusion "map" into actionable data name fix * Update psa.cpp Update psa.cpp * Update cvt.cpp * Update asvt.cpp * algorithm cleanup * better cleanup * Update control_block.cpp * Update main.cpp * Finish addressing merge issues and commit hooks -Minor code updates that were lost in merge commit -Bring new code up to standard for cppcheck -Format new code with clang-format * Update _cbxp.c * Streamline Control Block Explorer Class Update control_block_explorer.hpp * Massive refactor -Shave 2 step process down to one -Change serialized json inclusion map to use a vector of strings still split by "dot" operators * Error Handling Logic * BIG UPDATE PR COMMENTS -Switch pre-processing to one hash map function -use try/catch with custom errors rather than passing return codes everywhere -style and name changes -Enforce more rigid parm structure on entry -Fix some behavioral bugs and oversights in inclusion preprocessing -General streamlining and refactoring of functions, methods, classes, etc. * Another Big Refactor -PR comments (mostly style, but streamlining of error code as well) -Reworked base and derived classes to allow for includables to be defined to the base class and include_map to be defined to the base and derived classes * Update ascb.cpp * Update control_block.hpp * . * .. * ... * PR Comments -ASCB pointer deref in ASVT -Minor name changes -Remove double wildcard error -Add control_block_name_ private member and add initialization to constructor -move include_map_ to protected and remove private using statement * Update asvt.cpp * Update asvt.cpp * PR Comments Mostly renaming things streamlining some unnecessary text, parms and strings * Update control_block.cpp * Update main.cpp * Last round of PR comments string compare with == remove vestiges of old mechanisms for control block management name changes minor tweaks * Update cvt.cpp * Update cvt.cpp * Update cvt.cpp * Final comments Update control_block_explorer.cpp * comments * Last Comments * include changes * Last round of comments * debug * Unit testing (#17) * initial commit 1 * cleaned code before include test cases * wrote test cases, need to check with team now * added space after every function * added .value * shell script done * made changes proposed by leonard 1 * PR changes requested by team * added tests to check for ascb and asvt entries whether it be a string or dict * added tests to check for ascb and asvt entries whether it be a string or dict one more place * made minor tweaks * added updates provided by leonard * grouped failure test cases together * grouped error test cases together * removed extra lines * style changes * Feat/oss housekeeping2 (#18) * Set explicit C/C++ standard and cleanup README. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> * Update contribution guidelines and functional tests. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> * Cleanup contribution guidelines and debug debug mode. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> * Cleanup. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> * Cleanup. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> * Fix sdist packaging and pyproject.toml metadata. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> --------- Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> * Fix _C.pyi and removed unused import from cbxp.py. Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> --------- Signed-off-by: Leonard Carcaramo <lcarcaramo@ibm.com> Co-authored-by: Elijah Swift <Elijah.Swift@ibm.com> Co-authored-by: Varun Chennamadhava <varunchennamadhava@ibm.com>
|
DCO signoff seems to be missing on all commits. Maybe consider squashing all of these commits into one commit that has a DCO signoff? |
| void processAsteriskInclude(); | ||
| void processExplicitInclude(std::string& include); | ||
|
|
||
| protected: |
There was a problem hiding this comment.
We should restrict filters to only "repeated" control blocks. Maybe we can have a boolean class attribute to keep track of this?
const bool repeated_;There was a problem hiding this comment.
Should we? What if we have a script design to regularly run and throw flags if a certain value is found in a singular control block like the PSA? I see a much STRONGER benefit to using them in repeated control blocks, but I'm not convinced that they should be DISALLOWED from others.
| for (const auto& [filter_key, filter_value] : filter_map_) { | ||
| if (control_block_json.contains(filter_key)) { | ||
| try { | ||
| if (filter_value == control_block_json[filter_key].get<std::string>()) { |
There was a problem hiding this comment.
Should we consider trimming spaces from filter values for comparisons to make building filters a little less clunky?
./dist/cbxp -f assbjbni=USER assbThere was a problem hiding this comment.
If you want, you could also take a look at implementing wildcarding. I actually don't think this would be too difficult to implement. I think you can just use fnmatch() https://www.ibm.com/docs/en/zos/3.2.0?topic=functions-fnmatch-match-file-name-path-name
There was a problem hiding this comment.
It could also be worth starting to look into comparisons for other data types like int in particular. Maybe have support for >, <, >=, and <= comparisons:
cbxp -f "assb_time_on_cp>=50000" assbThere was a problem hiding this comment.
Yes, per our discussions I have been working on this. I have a way to do so, but it is based on "guess and check". I think as we explore adding more contextual data to the headers/json structures we use, including information on data typing would be helpful here.
| } else if (cbxp_result->result_json_length == null_length && | ||
| filter_key != "") { | ||
| std::cerr << "No control block was found that matched the provided filter" | ||
| << std::endl; |
There was a problem hiding this comment.
Should no matches result in an error or should we simply just return an empty JSON string and then the user can decide what to do with it from there?
There was a problem hiding this comment.
So I thought that returning an empty JSON with no context might somehow happen inadvertently in some other edge or error case. This is why I had it return an empty JSON, but have the CLI and Python interfaces treat it like an "error" anyway, rather than flagging this as an error in the main code. I am open to changes here, though.
Signed-off-by: Elijah Swift <Elijah.Swift@IBM.com>
ElijahSwiftIBM
force-pushed
the
feat/filter-control-blocks
branch
from
0d354a7 to
954478d
Compare
💡 Issue Reference
Issue: #5
💻 What does this address?
Control block filtering is a valuable enhancement. In basic cases it allows for simple validation that a control block containing the right data is collected, but in more complex cases, it allows users to grab information on just a singular relevant control block or collection of them in a large table.
📟 Implementation Details
The core control block class now has a new unordered map for filtration that is built on object creation. The filter is passed from block to block along with inclusion patterns and validated along the way. At the "end" of the line, the control block's value is validated against that of the filter. In the event of a match, the json data is returned, otherwise an empty json comes back. Large lists of control blocks like ASCB's and ASSB's were edited to filter out these null values, allowing for a true filter.
📋 Is there a test case?
I added the following tests in both the python and the shell test suites (the names in python are fare more descriptive)
test_cbxp_raises_cbxp_error_if_filter_uses_non_included_control_block
test_cbxp_raises_cbxp_error_if_filter_uses_unknown_key
test_cbxp_raises_cbxp_error_if_filter_dict_wrong_size (python only)
test_cbxp_raises_cbxp_error_if_filter_value_not_string (python only)
test_cbxp_raises_cbxp_error_if_no_filter_match
test_cbxp_can_use_basic_filter
test_cbxp_can_use_include_filter_with_generic_include
test_cbxp_can_use_include_filter_with_discrete_include