-
Notifications
You must be signed in to change notification settings - Fork 326
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Audit Logs for Postgres connections opened through a data link (#9873)
- Closes #9599 - Implemented API for sending audit logs to the cloud on a background thread. - If the Postgres connection is opened through a datalink, its internal JDBC connection is replaced by a wrapper that reports executed queries to the audit log. - Also introduces `EnsoMeta` - a helper Java class that can be used in our helper libraries to access Enso types. - I have replaced the common pattern scattered throughout the codebase with calls to this 'library' to avoid repetitive code. - Refactored `Table.display` to share code between in-memory and DB - it was needed as the function stopped working for `DB_Table` after adding making the `Table` constructor `private`. - Clearer error when reading a SQLite database from a remote file (tells the user to download it first). - Follow up - correlate asset id of the data link: #9869 - Follow up - include project name (once bug is fixed): #9875 - Some problems/improvements of the audit log: - The audit log system is not yet ready for high throughput of logs #9870 - The logs may be lost if `System.exit` is used #9871
- Loading branch information
Showing
45 changed files
with
2,175 additions
and
219 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
37 changes: 37 additions & 0 deletions
37
distribution/lib/Standard/Base/0.0.0-dev/src/Enso_Cloud/Internal/Audit_Log.enso
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| import project.Data.Json.JS_Object | ||
| import project.Data.Text.Text | ||
| import project.Errors.Illegal_Argument.Illegal_Argument | ||
| import project.Nothing.Nothing | ||
| from project.Data.Boolean import Boolean, False, True | ||
|
|
||
| polyglot java import org.enso.base.enso_cloud.audit.AuditLog | ||
|
|
||
| ## PRIVATE | ||
| type Audit_Log | ||
| ## PRIVATE | ||
| Reports an event to the audit log. | ||
| The event is submitted asynchronously. | ||
|
|
||
| Arguments: | ||
| - event_type: The type of the event. | ||
| - message: The message associated with the event. | ||
| - metadata: Additional metadata to include with the event. | ||
| Note that it should be a JS object and it should _not_ contain fields | ||
| that are restricted. These fields are added to the metadata | ||
| automatically. | ||
| - async: Whether to submit the event asynchronously. | ||
| Defaults to True. | ||
|
|
||
| ? Restricted Fields | ||
|
|
||
| The following fields are added by the system and should not be included | ||
| in the provided metadata: | ||
| - `type` | ||
| - `operation` | ||
| - `localTimestamp` | ||
| - `projectName` | ||
| report_event event_type:Text message:Text (metadata:JS_Object = JS_Object.from_pairs []) (async : Boolean = True) -> Nothing = | ||
| Illegal_Argument.handle_java_exception <| | ||
| case async of | ||
| True -> AuditLog.logAsync event_type message metadata.object_node | ||
| False -> AuditLog.logSynchronously event_type message metadata.object_node |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
89 changes: 89 additions & 0 deletions
89
distribution/lib/Standard/Table/0.0.0-dev/src/Internal/Display_Helpers.enso
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| from Standard.Base import all | ||
|
|
||
| import project.Table.Table | ||
| from project.Column import get_item_string, normalize_string_for_display | ||
|
|
||
| polyglot java import java.lang.System as Java_System | ||
|
|
||
| ## PRIVATE | ||
|
|
||
| Renders an ASCII-art representation for a Table from a dataframe that | ||
| contains a fragment of the underlying data and count of all rows. | ||
|
|
||
| Arguments: | ||
| - table: The materialized table that contains the data to be displayed. | ||
| - add_row_index: A boolean flag, specifying whether to display row indices. | ||
| - max_rows_to_show: The maximum amount of rows to display. | ||
| - all_rows_count: The count of all rows in the underlying Table; if | ||
| `all_rows_count` is bigger than the amount of rows displayed, an additional | ||
| line will be included that will say how many hidden rows there are. | ||
| Useful for remote tables where `df` contains only a fragment of the data. | ||
| - format_terminal: A boolean flag, specifying whether to use ANSI escape | ||
| codes for rich formatting in the terminal. | ||
| display_table (table : Table) (add_row_index : Boolean) (max_rows_to_show : Integer) (all_rows_count : Integer) (format_terminal : Boolean) -> Text = | ||
| cols = Vector.from_polyglot_array table.java_table.getColumns | ||
| col_names = cols.map .getName . map normalize_string_for_display | ||
| col_vals = cols.map .getStorage | ||
| display_rows = table.row_count.min max_rows_to_show | ||
| rows = Vector.new display_rows row_num-> | ||
| cols = col_vals.map col-> | ||
| if col.isNothing row_num then "Nothing" else get_item_string col row_num | ||
| if add_row_index then [row_num.to_text] + cols else cols | ||
| table_text = case add_row_index of | ||
| True -> print_table [""]+col_names rows 1 format_terminal | ||
| False -> print_table col_names rows 0 format_terminal | ||
| if display_rows == all_rows_count then table_text else | ||
| missing_rows_count = all_rows_count - display_rows | ||
| missing = '\n\u2026 and ' + missing_rows_count.to_text + ' hidden rows.' | ||
| table_text + missing | ||
|
|
||
| ## PRIVATE | ||
|
|
||
| A helper function for creating an ASCII-art representation of tabular data. | ||
|
|
||
| Arguments: | ||
| - header: vector of names of columns in the table. | ||
| - rows: a vector of rows, where each row is a vector that contains a text | ||
| representation of each cell | ||
| - indices_count: the number specifying how many columns should be treated as | ||
| indices; this will make them in bold font if `format_term` is enabled. | ||
| - format_term: a boolean flag, specifying whether to use ANSI escape codes | ||
| for rich formatting in the terminal. | ||
| print_table : Vector Text -> (Vector (Vector Text)) -> Integer -> Boolean -> Text | ||
| print_table header rows indices_count format_term = | ||
| content_lengths = Vector.new header.length i-> | ||
| max_row = 0.up_to rows.length . fold 0 a-> j-> a.max (rows.at j . at i . characters . length) | ||
| max_row.max (header.at i . characters . length) | ||
| header_line = header.zip content_lengths pad . map (ansi_bold format_term) . join ' | ' | ||
| divider = content_lengths . map (l -> "-".repeat l+2) . join '+' | ||
| row_lines = rows.map r-> | ||
| x = r.zip content_lengths pad | ||
| ixes = x.take (First indices_count) . map (ansi_bold format_term) | ||
| with_bold_ix = ixes + x.drop (First indices_count) | ||
| y = with_bold_ix . join ' | ' | ||
| " " + y | ||
| ([" " + header_line, divider] + row_lines).join '\n' | ||
|
|
||
| ## PRIVATE | ||
|
|
||
| Ensures that the `txt` has at least `len` characters by appending spaces at | ||
| the end. | ||
|
|
||
| Arguments: | ||
| - txt: The text to pad. | ||
| - len: The minimum length of the text. | ||
| pad : Text -> Integer -> Text | ||
| pad txt len = | ||
| true_len = txt.characters.length | ||
| txt + (" ".repeat (len - true_len)) | ||
|
|
||
| ## PRIVATE | ||
|
|
||
| Adds ANSI bold escape sequences to text if the feature is enabled. | ||
|
|
||
| Arguments: | ||
| - enabled: will insert ANSI sequences only if this flag is true and we are not on Windows. | ||
| - txt: The text to possibly bold. | ||
| ansi_bold : Boolean -> Text -> Text | ||
| ansi_bold enabled txt = | ||
| if enabled && (Java_System.console != Nothing) then '\e[1m' + txt + '\e[m' else txt |
Oops, something went wrong.