Skip to content

Commit 7e3892f

Browse files
KammerloM4rc0Russ0
Kammerlo
and
M4rc0Russ0
authored
Release 1.0.0 (#266)
* fix: unlock transactions which won't be published now (#264) * fix: unlock transactions which won't be published now * fix: unlock transactions which won't be published now * fix: unlock transactions which won't be published now * fix: adjusting metadataschema to accept strings or arrays when strings are to long * chore: fix tests * feat: unlocking remaining transactions * chore: adjusting tests * chore: adjusting blockchain publisher module to same number format as accounting core * fix: journal enrichment must adjust the operation type as well (#269) * fix: journal enrichment must adjust the operation type as well * chore: removing the workaround * chore: spotless * feat: added the possibility to adjust the start date for results carried forward * chore: adding tests * chore: adding tests * chore: spotless * chore: adjusting build job * feat: moving journalenrichmentstask and fixing totalLCY amount * Fix: Parent cost center missing in extraction * chore: fixing test * chore: renamed variable for debouncer configuration * fix: Missing currecny data should result invalid transaction * fix: transactional runner to not create new database transactions (#273) * fix: transactional runner to not create new database transactions * chore: adjusting the transactionBAtcHStatus update to run after transactions * chore: adjusting transaction starting * chore: test adjustment * chore: returned to save and flush * chore: spotless * chore: adding transactional back to update transaction batch * chore: spotless * fix: fixing transactional runner * fix: optimizing internal transactions (#276) * fix: optimizing internal transactions * fix: optimizing internal transactions * feat: adding TxStatusUpdaterJob * feat: adding TxStatusUpdaterJob * feat: adding TxStatusUpdaterJob * chore: adjusted test for new transactionUpdaterJob * chore: spotless * chore: refactored and more error save * chore: adding tests * fix: blockchain publisher didn't saved transaction when running in non kafka mode (#277) * fix: batches tx counting * fix: CARDANO_MAX_TRANSACTION_SIZE_BYTES transaction size * fix: extraction and public interface * fix: public service * feat: Report is set to pending if values don't match chore: renamed variable fix: null value on period * fix: account credit name * fix: revalt calculation fro CREDIT and DEBIT * fix: ERASED_SUM_APPLIED using account values * fix: txitem_group_strategy_new * fix: saving report status in db (#296) * fix: saving report status in db * chore: adding automated report updates * feat: only updating when the transaction is finalized * chore: changing query to only take not dispatched reports into account * Fix/save report status in db (#299) * fix: saving report status in db * chore: adding automated report updates * feat: only updating when the transaction is finalized * chore: changing query to only take not dispatched reports into account * chore: refactoring and error handling for already published reports * chore: spotless * Fix/use always latest report for metrics (#304) * fix: using only latest report for metrics * chore: adjusting tests for new behaviour * chore: report version readability * fix: total expense values were wrongly mapped (#307) * Feat/aggregate on publisher (#308) * feat: adding aggregation on publishing * chore: adding a comment to trigger the gha pipeline * Feat/aggregate Public Pages items (#310) * feat: aggregating items for public pages if they are displayed equally * chore: adding a comment to trigger gha * fix: change the amount reatment from Long to bigDecimal in the Public interface and also in the Publisher * fix: when TxType is FxReval * fix: rename the dummy account hard coded * chore: removing exception throw for null objects and skipping them (#316) * fix: report validation * feat: adding API versioning (#320) * Update onChainFormat.md (#329) * fix: avoiding to overwriting statuses (#326) * Update version in gradle.properties --------- Co-authored-by: Marco Russo <[email protected]> Co-authored-by: Marco Russo <[email protected]>
1 parent 41c4023 commit 7e3892f

File tree

81 files changed

+2020
-320
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+2020
-320
lines changed

‎.github/workflows/build.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@ name: Build
22

33
on:
44
push:
5-
branches: [ main, develop ]
5+
branches:
6+
- main
7+
- develop
8+
- 'release/*'
69
tags:
710
- '[0-9]+.[0-9]+.[0-9]+*'
811
pull_request:

‎_backend-services/cf-reeve-ledger-follower-app/src/main/java/org/cardano/foundation/lob/resource/LedgerFollowerResource.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030

3131
@RestController
3232
@CrossOrigin(origins = "http://localhost:3000")
33-
@RequestMapping("/api")
33+
@RequestMapping("/api/v1")
3434
@RequiredArgsConstructor
3535
@Slf4j
3636
public class LedgerFollowerResource {

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/config/BusinessRulesConfig.java‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ public BusinessRulesPipelineProcessor reprocessBusinessRulesProcessor() {
5656
val pipelineTasks = new ArrayList<PipelineTask>();
5757

5858
pipelineTasks.add(conversionPipelineTask());
59+
pipelineTasks.add(postCleansingPipelineTask());
5960

6061
return new DefaultBusinessRulesPipelineProcessor(pipelineTasks);
6162
}
@@ -69,8 +70,7 @@ private PipelineTask sanityCheckPipelineTask() {
6970

7071
private PipelineTask preCleansingPipelineTask() {
7172
return new DefaultPipelineTask(List.of(
72-
new DiscardZeroBalanceTxItemsTaskItem(),
73-
new JournalAccountCreditEnrichmentTaskItem(organisationPublicApi)
73+
new DiscardZeroBalanceTxItemsTaskItem()
7474
));
7575
}
7676

@@ -79,7 +79,8 @@ private PipelineTask preValidationPipelineTask() {
7979
new AmountsFcyCheckTaskItem(),
8080
new AmountsLcyCheckTaskItem(),
8181
new AmountLcyBalanceZerosOutCheckTaskItem(),
82-
new AmountFcyBalanceZerosOutCheckTaskItem()
82+
new AmountFcyBalanceZerosOutCheckTaskItem(),
83+
new JournalAccountCreditEnrichmentTaskItem(organisationPublicApi)
8384
));
8485
}
8586

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/domain/entity/report/ReportEntity.java‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,11 @@ public String getId() {
197197
@Setter
198198
private Boolean ledgerDispatchApproved = false;
199199

200+
@Column(name = "is_ready_to_publish", nullable = false)
201+
@Getter
202+
@Setter
203+
private Boolean isReadyToPublish = false;
204+
200205
@Column(name = "ledger_dispatch_status", nullable = false)
201206
@Enumerated(STRING)
202207
@JdbcType(PostgreSQLEnumJdbcType.class)

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/job/TxStatusUpdaterJob.java‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
package org.cardanofoundation.lob.app.accounting_reporting_core.job;
2+
3+
import java.time.LocalDate;
4+
import java.util.HashMap;
5+
import java.util.HashSet;
6+
import java.util.List;
7+
import java.util.Map;
8+
import java.util.Set;
9+
import java.util.concurrent.ConcurrentHashMap;
10+
11+
import lombok.RequiredArgsConstructor;
12+
import lombok.extern.slf4j.Slf4j;
13+
14+
import org.springframework.beans.factory.annotation.Value;
15+
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
16+
import org.springframework.scheduling.annotation.Scheduled;
17+
import org.springframework.stereotype.Service;
18+
import org.springframework.transaction.annotation.Transactional;
19+
20+
import io.vavr.control.Either;
21+
import org.zalando.problem.Problem;
22+
23+
import org.cardanofoundation.lob.app.accounting_reporting_core.domain.core.LedgerDispatchStatus;
24+
import org.cardanofoundation.lob.app.accounting_reporting_core.domain.core.TxStatusUpdate;
25+
import org.cardanofoundation.lob.app.accounting_reporting_core.domain.entity.TransactionEntity;
26+
import org.cardanofoundation.lob.app.accounting_reporting_core.domain.entity.report.ReportEntity;
27+
import org.cardanofoundation.lob.app.accounting_reporting_core.repository.ReportRepository;
28+
import org.cardanofoundation.lob.app.accounting_reporting_core.service.internal.LedgerService;
29+
import org.cardanofoundation.lob.app.accounting_reporting_core.service.internal.ReportService;
30+
import org.cardanofoundation.lob.app.accounting_reporting_core.service.internal.TransactionBatchService;
31+
32+
@Service
33+
@Slf4j
34+
@RequiredArgsConstructor
35+
@ConditionalOnProperty(value = "lob.accounting_reporting_core.enabled", havingValue = "true", matchIfMissing = true)
36+
public class TxStatusUpdaterJob {
37+
38+
private final Map<String, TxStatusUpdate> txStatusUpdatesMap = new ConcurrentHashMap<>();
39+
private final LedgerService ledgerService;
40+
private final TransactionBatchService transactionBatchService;
41+
private final ReportRepository reportRepository;
42+
private final ReportService reportService;
43+
44+
@Value("${lob.blockchain.tx-status-updater.max-map-size:1000}")
45+
private int maxMapSize;
46+
47+
// This Job collects all TxStatusUpdate events and updates the transactions in the database
48+
@Scheduled(
49+
fixedDelayString = "${ob.blockchain.tx-status-updater.fixed_delay:PT30S}",
50+
initialDelayString = "${lob.blockchain.tx-status-updater.delay:PT30S}")
51+
@Transactional
52+
public void execute() {
53+
Map<String, TxStatusUpdate> updates;
54+
synchronized (txStatusUpdatesMap) {
55+
updates = new HashMap<>(txStatusUpdatesMap);
56+
}
57+
if(updates.isEmpty()) {
58+
log.debug("No TxStatusUpdate events to process");
59+
return;
60+
}
61+
try {
62+
log.info("Updating Status of {} transactions", updates.size());
63+
List<TransactionEntity> transactionEntities = ledgerService.updateTransactionsWithNewStatuses(updates);
64+
ledgerService.saveAllTransactionEntities(transactionEntities);
65+
66+
transactionBatchService.updateBatchesPerTransactions(updates);
67+
updates.forEach(txStatusUpdatesMap::remove);
68+
69+
// Updating respective reports - Could be refactored to a separate method
70+
Set<ReportEntity> reportEntitiesToBeUpdated = new HashSet<>();
71+
for(TransactionEntity tx : transactionEntities) {
72+
if (tx.getLedgerDispatchStatus() == LedgerDispatchStatus.FINALIZED) {
73+
LocalDate date = tx.getEntryDate();
74+
int year = date.getYear();
75+
int month = date.getMonthValue();
76+
int quarter = (month - 1) / 3 + 1;
77+
reportEntitiesToBeUpdated.addAll(reportRepository.findNotPublishedByOrganisationIdAndContainingDate(tx.getOrganisation().getId(), year, quarter, month));
78+
}
79+
}
80+
81+
reportEntitiesToBeUpdated.forEach(report -> {
82+
log.info("Checking if report {} is ready to publish", report.getId());
83+
Either<Problem, Boolean> isReadyToPublish = reportService.canPublish(report);
84+
if (isReadyToPublish.isLeft()) {
85+
log.error("Report {} cannot be published: {}", report.getId(), isReadyToPublish.getLeft().getDetail());
86+
return;
87+
}
88+
report.setIsReadyToPublish(isReadyToPublish.get());
89+
reportRepository.save(report);
90+
});
91+
92+
} catch (Exception e) {
93+
log.error("Failed to process TxStatusUpdates - entries will be retained in the map", e);
94+
}
95+
}
96+
97+
public void addToStatusUpdateMap(Map<String, TxStatusUpdate> updateMap) {
98+
synchronized (txStatusUpdatesMap) {
99+
updateMap.forEach((key, value) -> {
100+
txStatusUpdatesMap.merge(key, value, (oldValue, newValue) ->
101+
newValue.getStatus().compareTo(oldValue.getStatus()) > 0 ? newValue : oldValue
102+
);
103+
});
104+
}
105+
if(txStatusUpdatesMap.size() > maxMapSize) {
106+
log.warn("TxStatusUpdate map size exceeded the limit of {}. Current size: {}", maxMapSize, txStatusUpdatesMap.size());
107+
}
108+
}
109+
110+
111+
}

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/repository/ReportRepository.java‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@
1515

1616
public interface ReportRepository extends JpaRepository<ReportEntity, String> {
1717

18+
Optional<ReportEntity> findFirstByOrganisationIdAndReportId(@Param("organisationId") String organisationId, @Param("reportId") String reportId);
19+
1820
@Query("""
1921
SELECT r FROM accounting_reporting_core.report.ReportEntity r
2022
WHERE r.organisation.id = :organisationId
@@ -75,4 +77,16 @@ List<ReportEntity> getNewestReportsInRange(@Param("organisationId") String organ
7577
AND r.ledgerDispatchStatus = 'FINALIZED'
7678
""")
7779
Set<ReportEntity> findByTypeAndWithinYearRange(@Param("organisationId") String organisationId, @Param("reportType") ReportType reportType, @Param("startYear") int startYear, @Param("endYear") int endYear);
80+
81+
@Query("""
82+
SELECT r FROM accounting_reporting_core.report.ReportEntity r
83+
WHERE
84+
r.organisation.id = :organisationId
85+
AND r.ledgerDispatchStatus = 'NOT_DISPATCHED'
86+
AND
87+
(r.intervalType = 'YEAR' AND r.year >= :year)
88+
OR (r.intervalType = 'QUARTER' AND ((r.year = :year AND r.period >= :quarter) OR (r.year > :year)))
89+
OR (r.intervalType = 'MONTH' AND ((r.year = :year AND r.period >= :month) OR (r.year > :year)))
90+
""")
91+
Set<ReportEntity> findNotPublishedByOrganisationIdAndContainingDate(@Param("organisationId") String organisationId, @Param("year") int year, @Param("quarter") int quarter, @Param("month") int month);
7892
}

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/resource/AccountingCoreResource.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646

4747
@RestController
4848
@CrossOrigin(origins = "http://localhost:3000")
49-
@RequestMapping("/api")
49+
@RequestMapping("/api/v1")
5050
@RequiredArgsConstructor
5151
@Slf4j
5252
@ConditionalOnProperty(value = "lob.accounting_reporting_core.enabled", havingValue = "true", matchIfMissing = true)

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/resource/AccountingCoreResourceNetSuiteMock.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717

1818
@RestController
1919
@CrossOrigin(origins = "http://localhost:3000")
20-
@RequestMapping("/api")
20+
@RequestMapping("/api/v1")
2121
@RequiredArgsConstructor
2222
@Slf4j
2323
@ConditionalOnProperty(value = "lob.accounting_reporting_core.enabled", havingValue = "true", matchIfMissing = true)

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/resource/AccountingCoreResourceReconciliation.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828

2929
@RestController
3030
@CrossOrigin(origins = "http://localhost:3000")
31-
@RequestMapping("/api")
31+
@RequestMapping("/api/v1")
3232
@RequiredArgsConstructor
3333
@Slf4j
3434
@ConditionalOnProperty(value = "lob.accounting_reporting_core.enabled", havingValue = "true", matchIfMissing = true)

‎accounting_reporting_core/src/main/java/org/cardanofoundation/lob/app/accounting_reporting_core/resource/ExperimentalAccountingCoreResource.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
import org.cardanofoundation.lob.app.accounting_reporting_core.service.internal.TransactionBatchService;
2424

2525
@RestController
26-
@RequestMapping("/api/core")
26+
@RequestMapping("/api/v1/core")
2727
@Slf4j
2828
@RequiredArgsConstructor
2929
@Deprecated(forRemoval = true)

0 commit comments

Comments
 (0)