Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(v1v2): refactor database queries for v1 and v2 #7244

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
d42cf45
refactor: restructure payment_methods_v2 migrations
hrithikesh026 Feb 6, 2025
36d96f6
fix: run v1 and v2 migrations separately
hrithikesh026 Feb 10, 2025
dcc0537
chore: drop unused integer id coloumn from tables
hrithikesh026 Feb 10, 2025
cbcb3d3
chore: remove id field from locker_mock_up
hrithikesh026 Feb 10, 2025
fced36d
chore: introduce id varchar in merchant_account
hrithikesh026 Feb 10, 2025
8bb3a61
chore: add product_type column in merchant_account
hrithikesh026 Feb 11, 2025
76fcb9c
Merge remote-tracking branch 'origin/main' into refactor_id_related_q…
hrithikesh026 Feb 11, 2025
ab96aee
update schema_v2.rs
hrithikesh026 Feb 11, 2025
1800d55
Merge branch 'main' into refactor_id_related_queries_v1_v2
hrithikesh026 Feb 11, 2025
35fbaa7
update schema_v2.rs
hrithikesh026 Feb 12, 2025
c1f0fda
change v2 diesel model
hrithikesh026 Feb 12, 2025
e4cfae4
Merge remote-tracking branch 'origin/main' into refactor_id_related_q…
hrithikesh026 Feb 12, 2025
16c3b58
populate id in merchant_account while inserting
hrithikesh026 Feb 12, 2025
7bdcc46
clone merchant_id
hrithikesh026 Feb 13, 2025
072e071
make revert back id columns to varchar instead of serial
hrithikesh026 Feb 13, 2025
782a963
delete remove_id.patch
hrithikesh026 Feb 13, 2025
bafd03a
Merge branch 'main' into refactor_id_related_queries_v1_v2
hrithikesh026 Feb 18, 2025
0e3d2d7
don't drop id for few tables
hrithikesh026 Feb 18, 2025
1952cad
add patch file
hrithikesh026 Feb 18, 2025
99931a3
remove commented code from migration file
hrithikesh026 Feb 18, 2025
04b5de7
Merge remote-tracking branch 'origin/main' into refactor_id_related_q…
hrithikesh026 Feb 19, 2025
68f4f48
update schema_v2.rs
hrithikesh026 Feb 19, 2025
5ff01f1
address v2 clippy errors
hrithikesh026 Feb 19, 2025
ddef69b
refactor v2 migrations to make v2 columns not null after introducing …
hrithikesh026 Feb 20, 2025
35d7bcf
update schema_v2.rs
hrithikesh026 Feb 20, 2025
854da0a
introduce separate migration to alter type of v1 columns
hrithikesh026 Feb 21, 2025
fbe3f1e
format v2 migrations
hrithikesh026 Feb 21, 2025
c57e4ff
add payer country back in down.sql
hrithikesh026 Feb 24, 2025
627ccf1
Merge remote-tracking branch 'origin/main' into refactor_id_related_q…
hrithikesh026 Feb 24, 2025
0c27611
Merge remote-tracking branch 'origin/main' into refactor_id_related_q…
hrithikesh026 Feb 25, 2025
87db6fe
chore: add appropriate revert queries in down.sql
hrithikesh026 Feb 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion crates/common_enums/src/enums.rs
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
mod accounts;
mod payments;
mod ui;
use std::num::{ParseFloatError, TryFromIntError};

pub use accounts::MerchantProductType;
pub use payments::ProductType;
use serde::{Deserialize, Serialize};
pub use ui::*;
use utoipa::ToSchema;

pub use super::connector_enums::RoutableConnectors;

#[doc(hidden)]
pub mod diesel_exports {
pub use super::{
Expand Down
28 changes: 28 additions & 0 deletions crates/common_enums/src/enums/accounts.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
use serde;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Unnecessary import, can be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am deriving serde::Deserialize and Serialize for enum ProductType

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm aware, you can directly use it in the derive() line, you don't need the explicit use serde; import.

use utoipa::ToSchema;
#[derive(
Default,
Clone,
Debug,
Comment on lines +3 to +6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This enum can also derive Copy.

Eq,
PartialEq,
serde::Deserialize,
serde::Serialize,
strum::Display,
strum::EnumString,
ToSchema,
Hash,
)]
#[router_derive::diesel_enum(storage_type = "text")]
#[serde(rename_all = "snake_case")]
#[strum(serialize_all = "snake_case")]
pub enum MerchantProductType {
Orchestration,
#[default]
Legacy,
Vault,
Recon,
Recovery,
CostObservability,
DynamicRouting,
}
66 changes: 66 additions & 0 deletions crates/diesel_models/drop_id.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
diff --git a/crates/diesel_models/src/schema.rs b/crates/diesel_models/src/schema.rs
index b35e809a0..749a5dd47 100644
--- a/crates/diesel_models/src/schema.rs
+++ b/crates/diesel_models/src/schema.rs
@@ -142,14 +142,13 @@ diesel::table! {
}

diesel::table! {
use diesel::sql_types::*;
use crate::enums::diesel_exports::*;

- blocklist_fingerprint (id) {
- id -> Int4,
+ blocklist_fingerprint (merchant_id, fingerprint_id) {
#[max_length = 64]
merchant_id -> Varchar,
#[max_length = 64]
fingerprint_id -> Varchar,
data_kind -> BlocklistDataKind,
encrypted_fingerprint -> Text,
@@ -158,14 +157,13 @@ diesel::table! {
}

diesel::table! {
use diesel::sql_types::*;
use crate::enums::diesel_exports::*;

- blocklist_lookup (id) {
- id -> Int4,
+ blocklist_lookup (merchant_id, fingerprint) {
#[max_length = 64]
merchant_id -> Varchar,
fingerprint -> Text,
}
}

@@ -300,13 +298,12 @@ diesel::table! {

diesel::table! {
use diesel::sql_types::*;
use crate::enums::diesel_exports::*;

configs (key) {
- id -> Int4,
#[max_length = 255]
key -> Varchar,
config -> Text,
}
}

@@ -615,14 +612,13 @@ diesel::table! {
}

diesel::table! {
use diesel::sql_types::*;
use crate::enums::diesel_exports::*;

- locker_mock_up (id) {
- id -> Int4,
+ locker_mock_up (card_id) {
#[max_length = 255]
card_id -> Varchar,
#[max_length = 255]
external_id -> Varchar,
#[max_length = 255]
card_fingerprint -> Varchar,
108 changes: 0 additions & 108 deletions crates/diesel_models/remove_id.patch

This file was deleted.

4 changes: 1 addition & 3 deletions crates/diesel_models/src/blocklist_fingerprint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,8 @@ pub struct BlocklistFingerprintNew {
#[derive(
Clone, Debug, Eq, PartialEq, Queryable, Identifiable, Selectable, Deserialize, Serialize,
)]
#[diesel(table_name = blocklist_fingerprint, check_for_backend(diesel::pg::Pg))]
#[diesel(table_name = blocklist_fingerprint, primary_key(merchant_id, fingerprint_id), check_for_backend(diesel::pg::Pg))]
pub struct BlocklistFingerprint {
#[serde(skip_serializing)]
pub id: i32,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cannot remove id from here right? This will not be backwards compatible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this will not be backwards compatible. Hence we need to go for a deployment with downtime. Instead of staggering.

pub merchant_id: common_utils::id_type::MerchantId,
pub fingerprint_id: String,
pub data_kind: common_enums::BlocklistDataKind,
Expand Down
4 changes: 1 addition & 3 deletions crates/diesel_models/src/blocklist_lookup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,8 @@ pub struct BlocklistLookupNew {
Deserialize,
Serialize,
)]
#[diesel(table_name = blocklist_lookup, check_for_backend(diesel::pg::Pg))]
#[diesel(table_name = blocklist_lookup, primary_key(merchant_id, fingerprint), check_for_backend(diesel::pg::Pg))]
pub struct BlocklistLookup {
#[serde(skip)]
pub id: i32,
pub merchant_id: common_utils::id_type::MerchantId,
pub fingerprint: String,
}
18 changes: 9 additions & 9 deletions crates/diesel_models/src/business_profile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -310,14 +310,6 @@ pub struct Profile {
pub always_collect_shipping_details_from_wallet_connector: Option<bool>,
pub tax_connector_id: Option<common_utils::id_type::MerchantConnectorAccountId>,
pub is_tax_connector_enabled: Option<bool>,
pub routing_algorithm_id: Option<common_utils::id_type::RoutingId>,
pub order_fulfillment_time: Option<i64>,
pub order_fulfillment_time_origin: Option<common_enums::OrderFulfillmentTimeOrigin>,
pub frm_routing_algorithm_id: Option<String>,
pub payout_routing_algorithm_id: Option<common_utils::id_type::RoutingId>,
pub default_fallback_routing: Option<pii::SecretSerdeValue>,
pub should_collect_cvv_during_payment: bool,
pub id: common_utils::id_type::ProfileId,
pub version: common_enums::ApiVersion,
pub dynamic_routing_algorithm: Option<serde_json::Value>,
pub is_network_tokenization_enabled: bool,
Expand All @@ -327,9 +319,17 @@ pub struct Profile {
pub is_click_to_pay_enabled: bool,
pub authentication_product_ids:
Option<common_types::payments::AuthenticationConnectorAccountMap>,
pub three_ds_decision_manager_config: Option<common_types::payments::DecisionManagerRecord>,
pub card_testing_guard_config: Option<CardTestingGuardConfig>,
pub card_testing_secret_key: Option<Encryption>,
pub routing_algorithm_id: Option<common_utils::id_type::RoutingId>,
pub order_fulfillment_time: Option<i64>,
pub order_fulfillment_time_origin: Option<common_enums::OrderFulfillmentTimeOrigin>,
pub frm_routing_algorithm_id: Option<String>,
pub payout_routing_algorithm_id: Option<common_utils::id_type::RoutingId>,
pub default_fallback_routing: Option<pii::SecretSerdeValue>,
pub three_ds_decision_manager_config: Option<common_types::payments::DecisionManagerRecord>,
pub should_collect_cvv_during_payment: bool,
pub id: common_utils::id_type::ProfileId,
}

impl Profile {
Expand Down
5 changes: 1 addition & 4 deletions crates/diesel_models/src/configs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,8 @@ pub struct ConfigNew {
}

#[derive(Default, Clone, Debug, Identifiable, Queryable, Selectable, Deserialize, Serialize)]
#[diesel(table_name = configs, check_for_backend(diesel::pg::Pg))]
#[diesel(table_name = configs, primary_key(key), check_for_backend(diesel::pg::Pg))]
pub struct Config {
#[serde(skip)]
pub id: i32,
pub key: String,
pub config: String,
}
Expand Down Expand Up @@ -49,7 +47,6 @@ impl From<ConfigUpdate> for ConfigUpdateInternal {
impl From<ConfigNew> for Config {
fn from(config_new: ConfigNew) -> Self {
Self {
id: 0i32,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cannot remove this as well right? this will cause downtime

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. We need to go with a downtime for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can add this in the patch for now, anyways we will need to have some downtime when deploying for v2, let's please add this in patch for now

key: config_new.key,
config: config_new.config,
}
Expand Down
3 changes: 1 addition & 2 deletions crates/diesel_models/src/locker_mock_up.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ use diesel::{Identifiable, Insertable, Queryable, Selectable};
use crate::schema::locker_mock_up;

#[derive(Clone, Debug, Eq, Identifiable, Queryable, Selectable, PartialEq)]
#[diesel(table_name = locker_mock_up, check_for_backend(diesel::pg::Pg))]
#[diesel(table_name = locker_mock_up, primary_key(card_id), check_for_backend(diesel::pg::Pg))]
pub struct LockerMockUp {
pub id: i32,
pub card_id: String,
pub external_id: String,
pub card_fingerprint: String,
Expand Down
Loading
Loading