From 4bfcd6804f388403ca74a02102b7a8a72fc63858 Mon Sep 17 00:00:00 2001 From: Tan Le Date: Wed, 5 May 2021 22:35:15 +1000 Subject: [PATCH 1/4] Add development documentation for audit events This guide provides an overview of how Audit Event works, and how to instrument new audit events. --- doc/development/audit_event_guide/index.md | 179 +++++++++++++++++++++ 1 file changed, 179 insertions(+) create mode 100644 doc/development/audit_event_guide/index.md diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md new file mode 100644 index 00000000000000..e188f43f866446 --- /dev/null +++ b/doc/development/audit_event_guide/index.md @@ -0,0 +1,179 @@ +--- +stage: Manage +group: Compliance +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Audit Event Guide + +This guide provides an overview of now Audit Event works, and how to instrument +new audit events. + +## What is Audit Event? + +Audit Events is a tool for GitLab owners and administrators to audit important +actions performed across the application. + +## Audit Event Schemas + +To instrument an audit event, the following attributes should be provided. + +| attribute | type | required | description | +|-----------|------|----------| ----------- | +| name | string | false | the action name to be audited, used for error tracking | +| author | User | true | the user who authors the change | +| scope | User, Project, Group | true | the scope which the audit event belongs to | +| target | Object | true | the target object being audited | +| ip_address | IPAddr | false | the request IP address | +| message | string | true | the message describing the action| + +## How to instrument new Audit Events + +There are currently three ways of instrumenting audit events. + +- Create a new class in `ee/lib/ee/audit/` and extend `AuditEventService` +- Call `AuditEventService` after a successful action +- Call `Gitlab::Audit::Auditor.audit` passing an action block + +This inconsistency leads to unexpected bugs, increases maintain effort and worsens +developer experience. Therefore, we propose to use `Gitlab::Audit::Auditor` to +instrument new audit events. + +With this new service, we can instrument audit events in two ways. + +### Using block to record multiple events + +This method is useful when events are emitted deep in the call stack. + +For example, we can record multiple audit events when the user updates a merge +request approval rule. As part of this user flow, we would like to audit changes +to both approvers and approval groups. In the initiating service +(i.e. `MergeRequestRuleUpdateService`), we can wrap the `execute` call as follows: + +```ruby +# in the initiating service +audit_context = { + name: 'merge_approval_rule_updated', + author: current_user, + scope: project_alpha, + target: merge_approval_rule, + ip_address: request.remote_ip, + message: 'Attempted to update an approval rule' +} + +Gitlab::Audit::Auditor.audit(audit_context) do + service.execute +end +``` + +In the model (i.e. `ApprovalProjectRule`), we can push audit events on model +callbacks (i.e. `after_save` or `after_add`). + +```ruby +# in the model +include Auditable + +def audit_add(model) + push_audit_event('Added an approver on Security rule') +end + +def audit_remove(model) + push_audit_event('Removed an approver on Security rule') +end +``` + +Please note that this method does not support actions that are asynchronous, or +span across multiple processes (i.e. background jobs). + +### Using standard method call to record single event + +This method allows recording single audit event, and involves less moving parts. + +```ruby +if merge_approval_rule.save + audit_context = { + name: 'merge_approval_rule_created', + author: current_user, + scope: project_alpha, + target: merge_approval_rule, + ip_address: request.remote_ip, + message: 'Created a new approval rule' + } + + Gitlab::Audit::Auditor.audit(audit_context) +end +``` + +## Audit Event instrumentation flows + +As aforementioned, we support 2 ways to instrument audit events. They +have different flows. + +### Using block to record multiple events + +We wrap the operation block in a `Gitlab::Audit::Auditor` which captures the +initial audit context (i.e. author, scope, target, ip_address) object that are +available at the time the operation is initiated. + +Extra instrumentation is required in the interacted classes in the chain with +`Auditable` mixin to add audit events to the Audit Event queue via `Gitlab::Audit::EventQueue`. + +The `EventQueue` are stored in local thread via `SafeRequestStore` and then later +extracted when we record an audit event in `Gitlab::Audit::Auditor`. + +```plantuml +skinparam shadowing false +skinparam BoxPadding 10 +skinparam ParticipantPadding 20 + +participant "Instrumented Class" as A +participant "Audit::Auditor" as A1 #LightBlue +participant "Audit::EventQueue" as B #LightBlue +participant "Interacted Class" as C +participant "AuditEvent" as D + +A->A1: audit { block } +activate A1 +A1->B: begin! +A1->C: block.call +activate A1 #FFBBBB +activate C +C-->B: push [ message ] +C-->A1: true +deactivate A1 +deactivate C +A1->B: read +activate A1 #FFBBBB +activate B +B-->A1: [ messages ] +deactivate B +A1->D: bulk_insert! +deactivate A1 +A1->B: end! +A1-->A: +deactivate A1 +``` + +### Using standard method call to record single event + +This method has a more straigh-forward flow, and does not rely on `EventQueue` +and local thread. + +```plantuml +skinparam shadowing false +skinparam BoxPadding 10 +skinparam ParticipantPadding 20 + +participant "Instrumented Class" as A +participant "Audit::Auditor" as B #LightBlue +participant "AuditEvent" as C + +A->B: audit +activate B +B->C: bulk_insert! +B-->A: +deactivate B +``` + +In addition to recording to the database, we also write these events to +[a log file](../../administration/logs.md#audit_jsonlog). -- GitLab From f3884c315f34800c3cff0102a820f63ba1cb4774 Mon Sep 17 00:00:00 2001 From: Max Woolf Date: Fri, 7 May 2021 01:25:44 +0000 Subject: [PATCH 2/4] Apply 5 suggestion(s) to 1 file(s) --- doc/development/audit_event_guide/index.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md index e188f43f866446..7266c8a8e20d67 100644 --- a/doc/development/audit_event_guide/index.md +++ b/doc/development/audit_event_guide/index.md @@ -6,17 +6,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Audit Event Guide -This guide provides an overview of now Audit Event works, and how to instrument +This guide provides an overview of how Audit Events work, and how to instrument new audit events. -## What is Audit Event? +## What are Audit Events? -Audit Events is a tool for GitLab owners and administrators to audit important +Audit Events are a tool for GitLab owners and administrators to view records of important actions performed across the application. ## Audit Event Schemas -To instrument an audit event, the following attributes should be provided. +To instrument an audit event, the following attributes should be provided: | attribute | type | required | description | |-----------|------|----------| ----------- | @@ -35,7 +35,7 @@ There are currently three ways of instrumenting audit events. - Call `AuditEventService` after a successful action - Call `Gitlab::Audit::Auditor.audit` passing an action block -This inconsistency leads to unexpected bugs, increases maintain effort and worsens +This inconsistency leads to unexpected bugs, increases maintainer effort and worsens the developer experience. Therefore, we propose to use `Gitlab::Audit::Auditor` to instrument new audit events. -- GitLab From 059de433f370ad000288f0cd319d49288562e79c Mon Sep 17 00:00:00 2001 From: Dan Jensen Date: Mon, 10 May 2021 21:33:17 +0000 Subject: [PATCH 3/4] Apply 1 suggestion(s) to 1 file(s) --- doc/development/audit_event_guide/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md index 7266c8a8e20d67..ce4eb794c4b701 100644 --- a/doc/development/audit_event_guide/index.md +++ b/doc/development/audit_event_guide/index.md @@ -156,7 +156,7 @@ deactivate A1 ### Using standard method call to record single event -This method has a more straigh-forward flow, and does not rely on `EventQueue` +This method has a more straight-forward flow, and does not rely on `EventQueue` and local thread. ```plantuml -- GitLab From 9d9a64680812cdd001623771340c4cb769dd2e40 Mon Sep 17 00:00:00 2001 From: Evan Read Date: Tue, 11 May 2021 06:05:13 +0000 Subject: [PATCH 4/4] Apply 11 suggestion(s) to 1 file(s) --- doc/development/audit_event_guide/index.md | 44 +++++++++++----------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md index ce4eb794c4b701..0bff297f2a0d3a 100644 --- a/doc/development/audit_event_guide/index.md +++ b/doc/development/audit_event_guide/index.md @@ -18,28 +18,31 @@ actions performed across the application. To instrument an audit event, the following attributes should be provided: -| attribute | type | required | description | -|-----------|------|----------| ----------- | -| name | string | false | the action name to be audited, used for error tracking | -| author | User | true | the user who authors the change | -| scope | User, Project, Group | true | the scope which the audit event belongs to | -| target | Object | true | the target object being audited | -| ip_address | IPAddr | false | the request IP address | -| message | string | true | the message describing the action| +| Attribute | Type | Required? | Description | +|:-------------|:---------------------|:----------|:----------------------------------------------------| +| `name` | string | false | Action name to be audited. Used for error tracking | +| `author` | User | true | User who authors the change | +| `scope` | User, Project, Group | true | Scope which the audit event belongs to | +| `target` | Object | true | Target object being audited | +| `ip_address` | IPAddr | false | Request IP address | +| `message` | string | true | Message describing the action | ## How to instrument new Audit Events -There are currently three ways of instrumenting audit events. +There are three ways of instrumenting audit events: - Create a new class in `ee/lib/ee/audit/` and extend `AuditEventService` - Call `AuditEventService` after a successful action - Call `Gitlab::Audit::Auditor.audit` passing an action block -This inconsistency leads to unexpected bugs, increases maintainer effort and worsens the -developer experience. Therefore, we propose to use `Gitlab::Audit::Auditor` to +This inconsistency leads to unexpected bugs, increases maintainer effort, and worsens the +developer experience. Therefore, we suggest you use `Gitlab::Audit::Auditor` to instrument new audit events. -With this new service, we can instrument audit events in two ways. +With new service, we can instrument audit events in two ways: + +- Using block for multiple events. +- Using standard method call for single events. ### Using block to record multiple events @@ -48,7 +51,7 @@ This method is useful when events are emitted deep in the call stack. For example, we can record multiple audit events when the user updates a merge request approval rule. As part of this user flow, we would like to audit changes to both approvers and approval groups. In the initiating service -(i.e. `MergeRequestRuleUpdateService`), we can wrap the `execute` call as follows: +(for example, `MergeRequestRuleUpdateService`), we can wrap the `execute` call as follows: ```ruby # in the initiating service @@ -66,8 +69,8 @@ Gitlab::Audit::Auditor.audit(audit_context) do end ``` -In the model (i.e. `ApprovalProjectRule`), we can push audit events on model -callbacks (i.e. `after_save` or `after_add`). +In the model (for example, `ApprovalProjectRule`), we can push audit events on model +callbacks (for example, `after_save` or `after_add`). ```ruby # in the model @@ -83,11 +86,11 @@ end ``` Please note that this method does not support actions that are asynchronous, or -span across multiple processes (i.e. background jobs). +span across multiple processes (for example, background jobs). ### Using standard method call to record single event -This method allows recording single audit event, and involves less moving parts. +This method allows recording single audit event and involves fewer moving parts. ```ruby if merge_approval_rule.save @@ -106,19 +109,18 @@ end ## Audit Event instrumentation flows -As aforementioned, we support 2 ways to instrument audit events. They -have different flows. +The two ways we can instrument audit events have different flows. ### Using block to record multiple events We wrap the operation block in a `Gitlab::Audit::Auditor` which captures the -initial audit context (i.e. author, scope, target, ip_address) object that are +initial audit context (that is, `author`, `scope`, `target`, `ip_address`) object that are available at the time the operation is initiated. Extra instrumentation is required in the interacted classes in the chain with `Auditable` mixin to add audit events to the Audit Event queue via `Gitlab::Audit::EventQueue`. -The `EventQueue` are stored in local thread via `SafeRequestStore` and then later +The `EventQueue` is stored in a local thread via `SafeRequestStore` and then later extracted when we record an audit event in `Gitlab::Audit::Auditor`. ```plantuml -- GitLab