Skip to content

Commit 46b8ed0

Browse files
mandy-chessellmandy-chessell
authored
Merge pull request #6150 from mandy-chessell/egeria-code
Archives and audit log component descriptions
2 parents f127ecb + 86d2824 commit 46b8ed0

File tree

160 files changed

+6020
-875
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

160 files changed

+6020
-875
lines changed

‎Content-Organization.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ The Egeria core repository contains the core Egeria functionality, and is organi
5252
* **[connector-configuration-factory](open-metadata-implementation/adapters/open-connectors/connector-configuration-factory)** creates **Connection** objects to configure the open connectors.
5353
* **[data-store-connectors](open-metadata-implementation/adapters/open-connectors/data-store-connectors)** contain OCF connectors to data stores on different data platforms.
5454
* **[discovery-service-connectors](open-metadata-implementation/adapters/open-connectors/discovery-service-connectors)** contain ODF discovery service implementations.
55+
* **[dynamic-archiver-connectors](open-metadata-implementation/adapters/open-connectors/dynamic-archiver-connectors)** contains dynamic archiver services implementations.
5556
* **[event-bus-connectors](open-metadata-implementation/adapters/open-connectors/event-bus-connectors)** supports different event/messaging infrastructures. They can be plugged into the topic connectors from the access-service-connectors and repository-service-connectors.
5657
* **[governance-action-connectors](open-metadata-implementation/adapters/open-connectors/governance-action-connectors)** contains GAF governance action implementations.
5758
* **[governance-daemon-connectors](open-metadata-implementation/adapters/open-connectors/governance-daemon-connectors)** contains connectors for the governance daemon servers that monitor activity or synchronize metadata and configuration asynchronously between different tools.
@@ -78,6 +79,7 @@ The Egeria core repository contains the core Egeria functionality, and is organi
7879

7980
* **[engine-services](open-metadata-implementation/engine-services)** - services that host data management and governance engines. These run in the [Engine Host](open-metadata-implementation/admin-services/docs/concepts/engine-host.md) OMAG Server.
8081
* **[asset-analysis](open-metadata-implementation/engine-services/asset-analysis)** - services that host [Open Discovery Framework (ODF)](open-metadata-implementation/frameworks/open-discovery-framework) discovery services.
82+
* **[archive-manager](open-metadata-implementation/engine-services/archive-manager)** - services that host dynamic archive services.
8183
* **[governance-action](open-metadata-implementation/engine-services/governance-action)** - services that host [Governance Action Framework (GAF)](open-metadata-implementation/frameworks/governance-action-framework) governance action services.
8284

8385
* **[frameworks](open-metadata-implementation/frameworks)** - frameworks that support pluggable components.
@@ -93,12 +95,15 @@ The Egeria core repository contains the core Egeria functionality, and is organi
9395
* **[open-lineage-services](open-metadata-implementation/governance-servers/open-lineage-services)** - provides historic warehouse for lineage.
9496

9597
* **[integration-services](open-metadata-implementation/integration-services)** - services that host integration connectors. These run in the [Integration Daemon](open-metadata-implementation/admin-services/docs/concepts/integration-daemon.md) OMAG Server.
98+
* **[analytics-integrator](open-metadata-implementation/integration-services/analytics-integrator)** - services to catalog analytics services.
9699
* **[api-integrator](open-metadata-implementation/integration-services/api-integrator)** - services to catalog APIs supported by API Managers.
97100
* **[catalog-integrator](open-metadata-implementation/integration-services/catalog-integrator)** - services to exchange metadata with asset managers and catalogs.
98101
* **[database-integrator](open-metadata-implementation/integration-services/database-integrator)** - services to capture metadata from file systems and file management applications.
99102
* **[files-integrator](open-metadata-implementation/integration-services/files-integrator)** - services to exchange metadata with relational databases.
100-
* **[organization-integrator](open-metadata-implementation/integration-services/organization-integrator)** - services to load metadata about the organization, including user identifiers, profiles, team structures and department hierarchy.
103+
* **[infrastructure-integrator](open-metadata-implementation/integration-services/infrastructure-integrator)** - services to describe deployed IT infrastructure as metadata.
101104
* **[lineage-integrator](open-metadata-implementation/integration-services/lineage-integrator)** - services to load lineage metadata.
105+
* **[organization-integrator](open-metadata-implementation/integration-services/organization-integrator)** - services to load metadata about the organization, including user identifiers, profiles, team structures and department hierarchy.
106+
* **[search-integrator](open-metadata-implementation/integration-services/search-integrator)** - services to maintain search indexes about open metadata.
102107
* **[security-integrator](open-metadata-implementation/integration-services/security-integrator)** - services to push security tags to an external security enforcement point.
103108
* **[topic-integrator](open-metadata-implementation/integration-services/topic-integrator)** - services to catalog topics and their event types from an Event Broker.
104109

‎content-packs/OpenMetadataTypes.json‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

‎open-metadata-distribution/open-metadata-assemblies/build.gradle‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ dependencies {
3838
implementation project(':open-metadata-implementation:adapters:open-connectors:integration-connectors:openlineage-integration-connectors')
3939
implementation project(':open-metadata-implementation:adapters:open-connectors:integration-connectors:elasticsearch-integration-connector')
4040
implementation project(':open-metadata-implementation:adapters:open-connectors:discovery-service-connectors')
41+
implementation project(':open-metadata-implementation:adapters:open-connectors:dynamic-archiver-connectors')
4142
implementation project(':open-metadata-implementation:adapters:open-connectors:governance-action-connectors')
4243
implementation project(':open-metadata-implementation:adapters:open-connectors:event-bus-connectors:open-metadata-topic-connectors:inmemory-open-metadata-topic-connector')
4344
implementation project(':open-metadata-implementation:adapters:open-connectors:event-bus-connectors:open-metadata-topic-connectors:kafka-open-metadata-topic-connector')
@@ -51,6 +52,7 @@ dependencies {
5152
implementation project(':open-metadata-resources:open-metadata-samples:access-services-samples:asset-management-samples:asset-reader-avro-sample')
5253
implementation project(':open-metadata-resources:open-metadata-samples:access-services-samples:asset-management-samples:asset-reader-csv-sample')
5354
implementation project(':open-metadata-resources:open-metadata-samples:access-services-samples:governance-program-client-samples:governance-leadership-sample')
55+
implementation project(':open-metadata-resources:open-metadata-samples:access-services-samples:governance-program-client-samples:governance-subject-area-sample')
5456
implementation project(':open-metadata-resources:open-metadata-samples:access-services-samples:governance-program-client-samples:governance-zone-create-sample')
5557
implementation project(':open-metadata-resources:open-metadata-samples:access-services-samples:subject-area-client-samples:subject-area-definition-sample')
5658
implementation project(':open-metadata-implementation:access-services:asset-catalog:asset-catalog-client')
@@ -129,6 +131,7 @@ distributions {
129131
from { project(':open-metadata-implementation:adapters:open-connectors:configuration-store-connectors:configuration-encrypted-file-store-connector').jar }
130132
from { project(':open-metadata-implementation:adapters:open-connectors:configuration-store-connectors:configuration-file-store-connector').jar }
131133
from { project(':open-metadata-implementation:adapters:open-connectors:discovery-service-connectors').jar }
134+
from { project(':open-metadata-implementation:adapters:open-connectors:dynamic-archiver-connectors').jar }
132135
from { project(':open-metadata-implementation:adapters:open-connectors:governance-action-connectors').jar }
133136
from { project(':open-metadata-implementation:adapters:open-connectors:integration-connectors:files-integration-connectors').jar }
134137
from { project(':open-metadata-implementation:adapters:open-connectors:integration-connectors:kafka-integration-connector').jar }
@@ -155,6 +158,7 @@ distributions {
155158
from { project(':open-metadata-resources:open-metadata-samples:access-services-samples:asset-management-samples:asset-reader-avro-sample').shadowJar }
156159
from { project(':open-metadata-resources:open-metadata-samples:access-services-samples:asset-management-samples:asset-reader-csv-sample').shadowJar }
157160
from { project(':open-metadata-resources:open-metadata-samples:access-services-samples:governance-program-client-samples:governance-leadership-sample').shadowJar }
161+
from { project(':open-metadata-resources:open-metadata-samples:access-services-samples:governance-program-client-samples:governance-subject-area-sample').shadowJar }
158162
from { project(':open-metadata-resources:open-metadata-samples:access-services-samples:governance-program-client-samples:governance-zone-create-sample').shadowJar }
159163
from { project(':open-metadata-resources:open-metadata-samples:access-services-samples:subject-area-client-samples:subject-area-definition-sample').shadowJar }
160164
from { project(':open-metadata-resources:open-metadata-samples:admin-services-samples:admin-services-config-metadata-server-sample').shadowJar }

‎open-metadata-distribution/open-metadata-assemblies/pom.xml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,10 @@
178178
<groupId>org.odpi.egeria</groupId>
179179
<artifactId>governance-leadership-sample</artifactId>
180180
</dependency>
181+
<dependency>
182+
<groupId>org.odpi.egeria</groupId>
183+
<artifactId>governance-subject-area-sample</artifactId>
184+
</dependency>
181185
<dependency>
182186
<groupId>org.odpi.egeria</groupId>
183187
<artifactId>governance-zone-create-sample</artifactId>

‎open-metadata-distribution/open-metadata-assemblies/src/main/assemblies/egeria-omag.xml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@
6666
<include>org.odpi.egeria:data-folder-connector:*</include>
6767
<include>org.odpi.egeria:discovery-service-connectors:*</include>
6868
<include>org.odpi.egeria:governance-action-connectors:*</include>
69+
<include>org.odpi.egeria:dynamic-archiver-connectors:*</include>
6970
<include>org.odpi.egeria:files-integration-connectors:*</include>
7071
<include>org.odpi.egeria:kafka-integration-connector:*</include>
7172
<include>org.odpi.egeria:openapi-integration-connector:*</include>
@@ -137,6 +138,7 @@
137138
<include>org.odpi.egeria:asset-reader-avro-sample:*</include>
138139
<include>org.odpi.egeria:asset-reader-csv-sample:*</include>
139140
<include>org.odpi.egeria:governance-leadership-sample:*</include>
141+
<include>org.odpi.egeria:governance-subject-area-sample:*</include>
140142
<include>org.odpi.egeria:governance-zone-create-sample:*</include>
141143
<include>org.odpi.egeria:subject-area-definition-sample:*</include>
142144
</includes>

‎open-metadata-implementation/access-services/asset-catalog/asset-catalog-client/src/main/java/org/odpi/openmetadata/accessservices/assetcatalog/eventclient/AssetCatalogEventClient.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ public void registerListener(String userId,
161161
callerId);
162162

163163
Connection topicConnection = restResult.getConnection();
164-
ConnectorBroker connectorBroker = new ConnectorBroker();
164+
ConnectorBroker connectorBroker = new ConnectorBroker(auditLog);
165165
Connector connector = connectorBroker.getConnector(topicConnection);
166166

167167
if (connector == null) {

‎open-metadata-implementation/access-services/asset-consumer/asset-consumer-api/src/main/java/org/odpi/openmetadata/accessservices/assetconsumer/ffdc/AssetConsumerAuditCode.java‎

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,13 @@ public enum AssetConsumerAuditCode implements AuditLogMessageSet
3232
"If it is not required, then remove the configuration for this service " +
3333
"from the access service list in this server's configuration document."),
3434

35+
SERVICE_PUBLISHING("OMAS-ASSET-CONSUMER-0002",
36+
OMRSAuditLogRecordSeverity.STARTUP,
37+
"The Asset Consumer Open Metadata Access Service (OMAS) is ready to publish asset notifications to topic {0}",
38+
"The local server has started up the event publisher for the Asset Consumer OMAS. " +
39+
"It will begin publishing asset configuration changes to its out topic.",
40+
"This is part of the normal start up of the service. Check that there are no errors from the event bus."),
41+
3542
SERVICE_INITIALIZED("OMAS-ASSET-CONSUMER-0003",
3643
OMRSAuditLogRecordSeverity.STARTUP,
3744
"The Asset Consumer Open Metadata Access Service (OMAS) has initialized a new instance for server {0}",
@@ -59,13 +66,42 @@ public enum AssetConsumerAuditCode implements AuditLogMessageSet
5966
"An asset consumer has logged a message for an asset.",
6067
"Review the message to ensure no action is required."),
6168

69+
PUBLISHING_SHUTDOWN("OMAS-ASSET-CONSUMER-0010",
70+
OMRSAuditLogRecordSeverity.SHUTDOWN,
71+
"The Asset Consumer Open Metadata Access Service (OMAS) is no longer publishing events to topic {0}",
72+
"The local administrator has requested shut down of an Asset Consumer OMAS instance. " +
73+
"No more configuration events will be published to the named topic.",
74+
"This is part of the normal shutdown of the service. No action is required if this is service" +
75+
"shutdown was intentional."),
76+
77+
PUBLISHING_SHUTDOWN_ERROR("OMAS-ASSET-CONSUMER-0011",
78+
OMRSAuditLogRecordSeverity.SHUTDOWN,
79+
"The Asset Consumer Open Metadata Access Service (OMAS) caught an unexpected {0} exception whilst shutting down the out " +
80+
"topic {1}. The error message was: {2}",
81+
"The local administrator has requested shut down of an Asset Consumer OMAS instance. " +
82+
"No more configuration events will be published to the named topic, although the connection to the event bus may " +
83+
"not be released properly.",
84+
"This is part of the normal shutdown of the service. However, an exception is not expected at this point unless it " +
85+
"is the consequence of a previous error. Review the error message and any other reported failures to " +
86+
"determine if this exception needs special attention."),
87+
6288
OUT_TOPIC_EVENT("OMAS-ASSET-CONSUMER-0012",
6389
OMRSAuditLogRecordSeverity.EVENT,
64-
"The Asset Consumer Open Metadata Access Service (OMAS) has sent event: {0}",
90+
"The Asset Consumer Open Metadata Access Service (OMAS) has sent event of type: {0}",
6591
"The access service sends out asset notifications to ensure connected tools have the most up to-date " +
6692
"knowledge about assets. This audit log message is to create a record of the events that are being published.",
6793
"This event indicates that the metadata for an asset has changed. This my or may not be significant to " +
68-
"the receiving tools.") ;
94+
"the receiving tools."),
95+
96+
OUT_TOPIC_FAILURE("OMAS-ASSET-CONSUMER-0018",
97+
OMRSAuditLogRecordSeverity.EXCEPTION,
98+
"The Asset Consumer Open Metadata Access Service (OMAS) is unable to send an event on its out topic {0}; exception {1} returned " +
99+
"error message: {2}",
100+
"The access service detected an error during the start up of the out topic. Its services are not available for the server.",
101+
"Review the error message and any other reported failures to determine the cause of the problem. Check the status of the event " +
102+
"bus. Once this is resolved, restart the server."),
103+
104+
;
69105

70106
private static final long serialVersionUID = 1L;
71107

‎open-metadata-implementation/access-services/asset-consumer/asset-consumer-client/src/main/java/org/odpi/openmetadata/accessservices/assetconsumer/client/AssetConsumerEventClient.java‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ public void registerListener(String userId,
134134
callerId);
135135

136136
Connection topicConnection = restResult.getConnection();
137-
ConnectorBroker connectorBroker = new ConnectorBroker();
137+
ConnectorBroker connectorBroker = new ConnectorBroker(auditLog);
138138
Connector connector = connectorBroker.getConnector(topicConnection);
139139

140140
if (connector == null)
@@ -150,7 +150,6 @@ public void registerListener(String userId,
150150
if (connector instanceof AssetConsumerOutTopicClientConnector)
151151
{
152152
configurationEventTopicConnector = (AssetConsumerOutTopicClientConnector)connector;
153-
configurationEventTopicConnector.setAuditLog(auditLog);
154153
configurationEventTopicConnector.start();
155154
}
156155
else

‎open-metadata-implementation/access-services/asset-consumer/asset-consumer-server/pom.xml‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,7 @@
3030

3131
<dependencies>
3232

33-
<dependency>
34-
<groupId>com.fasterxml.jackson.core</groupId>
35-
<artifactId>jackson-databind</artifactId>
36-
</dependency>
37-
38-
<dependency>
33+
<dependency>
3934
<groupId>org.odpi.egeria</groupId>
4035
<artifactId>repository-services-apis</artifactId>
4136
</dependency>

0 commit comments

Comments
 (0)