diff --git a/spp_hazard_programs/README.rst b/spp_hazard_programs/README.rst index b098c5c0..51f5425f 100644 --- a/spp_hazard_programs/README.rst +++ b/spp_hazard_programs/README.rst @@ -1,18 +1,22 @@ +.. image:: https://odoo-community.org/readme-banner-image + :target: https://odoo-community.org/get-involved?utm_source=readme + :alt: Odoo Community Association + =================================== OpenSPP Hazard Programs Integration =================================== -.. +.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !! This file is generated by oca-gen-addon-readme !! !! changes will be overwritten. !! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! - !! source digest: sha256:e4ca3f5b5e8c998b833ccd6526f575c07f53a38ac49875511c31201f13122916 + !! source digest: sha256:4af3e4daf1509efef99ad29497bc756e1ec197df2297c50529ff48112fb08d59 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -.. |badge1| image:: https://img.shields.io/badge/maturity-Alpha-red.png +.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png :target: https://odoo-community.org/page/development-status - :alt: Alpha + :alt: Beta .. |badge2| image:: https://img.shields.io/badge/license-LGPL--3-blue.png :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html :alt: License: LGPL-3 @@ -44,15 +48,15 @@ Key Capabilities Key Models ~~~~~~~~~~ -+----------------------------------+-----------------------------------+ -| Model | Description | -+==================================+===================================+ -| ``spp.program`` (extend) | Adds target incidents, emergency | -| | mode, damage filter | -+----------------------------------+-----------------------------------+ -| ``spp.hazard.incident`` (extend) | Adds reverse relation to response | -| | programs | -+----------------------------------+-----------------------------------+ ++----------------------------------+----------------------------------+ +| Model | Description | ++==================================+==================================+ +| ``spp.program`` (extend) | Adds target incidents, emergency | +| | mode, damage filter | ++----------------------------------+----------------------------------+ +| ``spp.hazard.incident`` (extend) | Adds reverse relation to | +| | response programs | ++----------------------------------+----------------------------------+ UI Location ~~~~~~~~~~~ @@ -89,10 +93,6 @@ Dependencies ``spp_hazard``, ``spp_programs`` -.. IMPORTANT:: - This is an alpha version, the data model and design can change at any time without warning. - Only for development or testing purpose, do not use in production. - **Table of contents** .. contents:: @@ -135,4 +135,4 @@ Current maintainers: This module is part of the `OpenSPP/OpenSPP2 `_ project on GitHub. -You are welcome to contribute. \ No newline at end of file +You are welcome to contribute. diff --git a/spp_hazard_programs/__manifest__.py b/spp_hazard_programs/__manifest__.py index d81d7b17..40e9f9da 100644 --- a/spp_hazard_programs/__manifest__.py +++ b/spp_hazard_programs/__manifest__.py @@ -12,14 +12,13 @@ "author": "OpenSPP.org", "website": "https://github.com/OpenSPP/OpenSPP2", "license": "LGPL-3", - "development_status": "Alpha", + "development_status": "Beta", "maintainers": ["jeremi", "gonzalesedwin1123", "reichie020212"], "depends": [ "spp_hazard", "spp_programs", ], "data": [ - "security/ir.model.access.csv", "views/program_views.xml", "views/incident_views.xml", ], diff --git a/spp_hazard_programs/security/ir.model.access.csv b/spp_hazard_programs/security/ir.model.access.csv deleted file mode 100644 index 97dd8b91..00000000 --- a/spp_hazard_programs/security/ir.model.access.csv +++ /dev/null @@ -1 +0,0 @@ -id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink diff --git a/spp_hazard_programs/static/description/index.html b/spp_hazard_programs/static/description/index.html index d86761dd..732c50ae 100644 --- a/spp_hazard_programs/static/description/index.html +++ b/spp_hazard_programs/static/description/index.html @@ -3,7 +3,7 @@ -OpenSPP Hazard Programs Integration +README.rst -
-

OpenSPP Hazard Programs Integration

+
+ + +Odoo Community Association + +
+

OpenSPP Hazard Programs Integration

-

Alpha License: LGPL-3 OpenSPP/OpenSPP2

+

Beta License: LGPL-3 OpenSPP/OpenSPP2

Links hazard incidents to emergency response programs. Enables programs to target affected populations using verified impact data, filter registrants by damage severity, and automatically enable emergency mode when responding to active incidents.

-

Key Capabilities

+

Key Capabilities

  • Link programs to one or more hazard incidents via many-to-many relation
  • @@ -390,11 +395,11 @@

    Key Capabilities

-

Key Models

+

Key Models

--++ @@ -407,14 +412,14 @@

Key Models

mode, damage filter - +
Model
spp.hazard.incident (extend)Adds reverse relation to response -programsAdds reverse relation to +response programs
-

UI Location

+

UI Location

  • Programs: Programs > Programs > “Emergency Response” tab
  • Incidents: Hazard & Emergency > Incidents > All Incidents > @@ -426,7 +431,7 @@

    UI Location

-

Security

+

Security

No new ACL entries. Access inherited from base models:

  • spp.program: Controlled by spp_programs security groups
  • @@ -434,7 +439,7 @@

    Security

-

Extension Points

+

Extension Points

  • Override get_emergency_eligible_registrants() to customize eligibility logic beyond damage levels
  • @@ -446,13 +451,8 @@

    Extension Points

-

Dependencies

+

Dependencies

spp_hazard, spp_programs

-
-

Important

-

This is an alpha version, the data model and design can change at any time without warning. -Only for development or testing purpose, do not use in production.

-

Table of contents

    @@ -465,7 +465,7 @@

    Dependencies

-

Bug Tracker

+

Bug Tracker

Bugs are tracked on GitHub Issues. In case of trouble, please check there if your issue has already been reported. If you spotted it first, help us to smash it by providing a detailed and welcomed @@ -473,15 +473,15 @@

Bug Tracker

Do not contact contributors directly about support or help with technical issues.

-

Credits

+

Credits

-

Authors

+

Authors

  • OpenSPP.org
-

Maintainers

+

Maintainers

Current maintainers:

jeremi gonzalesedwin1123 reichie020212

This module is part of the OpenSPP/OpenSPP2 project on GitHub.

@@ -490,5 +490,6 @@

Maintainers

+
diff --git a/spp_hazard_programs/views/program_views.xml b/spp_hazard_programs/views/program_views.xml index 56ece548..935174da 100644 --- a/spp_hazard_programs/views/program_views.xml +++ b/spp_hazard_programs/views/program_views.xml @@ -63,7 +63,6 @@ - diff --git a/spp_import_match/README.rst b/spp_import_match/README.rst index c35f1ede..19904698 100644 --- a/spp_import_match/README.rst +++ b/spp_import_match/README.rst @@ -1,18 +1,22 @@ +.. image:: https://odoo-community.org/readme-banner-image + :target: https://odoo-community.org/get-involved?utm_source=readme + :alt: Odoo Community Association + ==================== OpenSPP Import Match ==================== -.. +.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !! This file is generated by oca-gen-addon-readme !! !! changes will be overwritten. !! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! - !! source digest: sha256:b57dea1315fd1f9af8f15720c5332b182c0a01fddd827a3daa73a8e950d41faa + !! source digest: sha256:02de7214923a9a8bd540920bb2ff0166ea13501c1859b831f3a8305970b5d64a !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -.. |badge1| image:: https://img.shields.io/badge/maturity-Alpha-red.png +.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png :target: https://odoo-community.org/page/development-status - :alt: Alpha + :alt: Beta .. |badge2| image:: https://img.shields.io/badge/license-LGPL--3-blue.png :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html :alt: License: LGPL-3 @@ -47,15 +51,15 @@ Key Capabilities Key Models ~~~~~~~~~~ -+-----------------------------+----------------------------------------+ -| Model | Description | -+=============================+========================================+ -| ``spp.import.match`` | Matching rule configuration for a | -| | specific model | -+-----------------------------+----------------------------------------+ -| ``spp.import.match.fields`` | Individual fields used in a rule, | -| | supports sub-fields | -+-----------------------------+----------------------------------------+ ++-----------------------------+---------------------------------------+ +| Model | Description | ++=============================+=======================================+ +| ``spp.import.match`` | Matching rule configuration for a | +| | specific model | ++-----------------------------+---------------------------------------+ +| ``spp.import.match.fields`` | Individual fields used in a rule, | +| | supports sub-fields | ++-----------------------------+---------------------------------------+ Configuration ~~~~~~~~~~~~~ @@ -106,10 +110,6 @@ Dependencies ``base``, ``spp_base_common``, ``base_import``, ``queue_job``, ``spp_security`` -.. IMPORTANT:: - This is an alpha version, the data model and design can change at any time without warning. - Only for development or testing purpose, do not use in production. - **Table of contents** .. contents:: @@ -149,4 +149,4 @@ Current maintainers: This module is part of the `OpenSPP/OpenSPP2 `_ project on GitHub. -You are welcome to contribute. \ No newline at end of file +You are welcome to contribute. diff --git a/spp_import_match/__manifest__.py b/spp_import_match/__manifest__.py index aca536e8..d12d3578 100644 --- a/spp_import_match/__manifest__.py +++ b/spp_import_match/__manifest__.py @@ -10,7 +10,7 @@ "author": "OpenSPP.org", "website": "https://github.com/OpenSPP/OpenSPP2", "license": "LGPL-3", - "development_status": "Alpha", + "development_status": "Beta", "maintainers": ["jeremi", "gonzalesedwin1123"], "depends": ["base", "spp_base_common", "base_import", "queue_job", "spp_security"], "data": [ @@ -20,10 +20,8 @@ ], "assets": { "web.assets_backend": [ - "spp_import_match/static/src/legacy/js/custom_base_import.js", - ], - "web.assets_qweb": [ - "spp_import_match/static/src/legacy/xml/custom_base_import.xml", + "spp_import_match/static/src/import_match_selector/import_match_selector.js", + "spp_import_match/static/src/import_match_selector/import_match_selector.xml", ], }, "demo": [], diff --git a/spp_import_match/models/base.py b/spp_import_match/models/base.py index 93faebe6..bf120686 100644 --- a/spp_import_match/models/base.py +++ b/spp_import_match/models/base.py @@ -1,10 +1,14 @@ # Part of OpenSPP. See LICENSE file for full copyright and licensing details. import logging +import threading from odoo import api, models _logger = logging.getLogger(__name__) +# Thread-local storage to pass import match counts from load() to execute_import() +_import_match_local = threading.local() + class Base(models.AbstractModel): _inherit = "base" @@ -16,14 +20,20 @@ def load(self, fields, data): fields, option_config_ids=self.env.context.get("import_match_ids", []), ) - model_id = self.env["ir.model"].search([("model", "=", self._name)]) - overwrite_match = True - import_match = self.env["spp.import.match"].search([("model_id", "=", model_id.id)]) - if import_match: - overwrite_match = import_match.overwrite_match + # If overwrite_match is explicitly set via UI (context), use that. + # Otherwise fall back to config records' overwrite_match setting. + if "overwrite_match" in self.env.context: + overwrite_match = self.env.context["overwrite_match"] + elif usable: + overwrite_match = any(self.env["spp.import.match"].browse(usable).mapped("overwrite_match")) + else: + overwrite_match = False if usable: newdata = list() + match_created = 0 + match_skipped = 0 + match_overwritten = 0 if ".id" in fields: column = fields.index(".id") fields[column] = "id" @@ -70,6 +80,7 @@ def load(self, fields, data): row["id"] = ext_id[match.id] if match else row.get("id", "") if match: if overwrite_match: + match_overwritten += 1 flat_fields_to_remove = [item for sublist in field_to_match for item in sublist] for fields_pop in flat_fields_to_remove: # Set one2many and many2many fields to False if matched @@ -80,19 +91,25 @@ def load(self, fields, data): ]: row[fields_pop] = False newdata.append(tuple(row[f] for f in clean_fields)) + else: + match_skipped += 1 else: + match_created += 1 newdata.append(tuple(row[f] for f in fields)) data = newdata + if self.env.context.get("import_match_ids"): + _import_match_local.counts = { + "created": match_created, + "skipped": match_skipped, + "overwritten": match_overwritten, + } return super().load(fields, data) def write(self, vals): - # nosemgrep: odoo-sudo-without-context - reading model metadata requires sudo - model = self.env["ir.model"].sudo().search([("model", "=", self._name)]) new_vals = vals.copy() - for rec in vals: - field_name = rec - if not vals[field_name]: - field = self.env["ir.model.fields"].search([("model_id", "=", model.id), ("name", "=", field_name)]) - if field and field.ttype in ("one2many", "many2many"): - new_vals.pop(rec) + for field_name, value in vals.items(): + if not value and field_name in self._fields: + field_meta = self._fields[field_name] + if field_meta.type in ("one2many", "many2many"): + new_vals.pop(field_name) return super().write(new_vals) diff --git a/spp_import_match/models/base_import.py b/spp_import_match/models/base_import.py index 2ac91bde..5f3ac8ea 100644 --- a/spp_import_match/models/base_import.py +++ b/spp_import_match/models/base_import.py @@ -9,6 +9,8 @@ from odoo.addons.queue_job.exception import FailedJobError +from .base import _import_match_local + _logger = logging.getLogger(__name__) # options defined in base_import/import.js OPT_HAS_HEADER = "headers" @@ -62,18 +64,19 @@ def execute_import(self, fields, columns, options, dryrun=False): _logger.info("Started Import: %s with rows %d", self.res_model, len(input_file_data)) import_match_ids = options.get("import_match_ids", []) + overwrite_match = options.get("overwrite_match", False) + _import_match_local.counts = None - if dryrun: - _logger.info("Doing dry-run import") - if import_match_ids: - self = self.with_context(import_match_ids=import_match_ids) - return super().execute_import(fields, columns, options, dryrun=True) - - if len(input_file_data) <= 100: - _logger.info("Doing normal import") + if dryrun or len(input_file_data) <= 100: + _logger.info("Doing %s import", "dry-run" if dryrun else "normal") if import_match_ids: - self = self.with_context(import_match_ids=import_match_ids) - return super().execute_import(fields, columns, options, dryrun=False) + self = self.with_context(import_match_ids=import_match_ids, overwrite_match=overwrite_match) + result = super().execute_import(fields, columns, options, dryrun=dryrun) + counts = getattr(_import_match_local, "counts", None) + if counts: + result["import_match_counts"] = counts + _import_match_local.counts = None + return result _logger.info("Started Asynchronous Import: %s", self.res_model) # asynchronous import diff --git a/spp_import_match/models/import_match.py b/spp_import_match/models/import_match.py index cdaef4bf..fc78a91d 100644 --- a/spp_import_match/models/import_match.py +++ b/spp_import_match/models/import_match.py @@ -70,7 +70,7 @@ def _match_find(self, model, converted_row, imported_row): if len(match) == 1: return match elif len(match) > 1: - raise ValidationError(_("Multiple matches found for '%s'!").format(match[0].name)) + raise ValidationError(_("Multiple matches found for '%s'!") % match[0].name) return model diff --git a/spp_import_match/static/description/index.html b/spp_import_match/static/description/index.html index b3369e2e..e6d71ccc 100644 --- a/spp_import_match/static/description/index.html +++ b/spp_import_match/static/description/index.html @@ -3,7 +3,7 @@ -OpenSPP Import Match +README.rst -
-

OpenSPP Import Match

+
+ + +Odoo Community Association + +
+

OpenSPP Import Match

-

Alpha License: LGPL-3 OpenSPP/OpenSPP2

+

Beta License: LGPL-3 OpenSPP/OpenSPP2

Extends Odoo’s base import functionality to match incoming records against existing data during bulk imports. Prevents duplicate creation by comparing imported rows to database records using configurable field combinations. Supports overwriting matched records and asynchronous processing for large datasets.

-

Key Capabilities

+

Key Capabilities

  • Define matching rules per model using field combinations to identify existing records
  • @@ -393,11 +398,11 @@

    Key Capabilities

-

Key Models

+

Key Models

--++ @@ -417,7 +422,7 @@

Key Models

Model
-

Configuration

+

Configuration

After installing:

  1. Navigate to Registry > Configuration > Import Match
  2. @@ -432,7 +437,7 @@

    Configuration

-

UI Location

+

UI Location

  • Menu: Registry > Configuration > Import Match
  • Import Dialog: Matching applies automatically during CSV import @@ -442,7 +447,7 @@

    UI Location

-

Security

+

Security

@@ -461,7 +466,7 @@

Security

-

Extension Points

+

Extension Points

  • Override spp.import.match._match_find() to customize matching logic for specific use cases
  • @@ -472,14 +477,9 @@

    Extension Points

-

Dependencies

+

Dependencies

base, spp_base_common, base_import, queue_job, spp_security

-
-

Important

-

This is an alpha version, the data model and design can change at any time without warning. -Only for development or testing purpose, do not use in production.

-

Table of contents

    @@ -492,7 +492,7 @@

    Dependencies

-

Bug Tracker

+

Bug Tracker

Bugs are tracked on GitHub Issues. In case of trouble, please check there if your issue has already been reported. If you spotted it first, help us to smash it by providing a detailed and welcomed @@ -500,15 +500,15 @@

Bug Tracker

Do not contact contributors directly about support or help with technical issues.

-

Credits

+

Credits

-

Authors

+

Authors

  • OpenSPP.org
-

Maintainers

+

Maintainers

Current maintainers:

jeremi gonzalesedwin1123

This module is part of the OpenSPP/OpenSPP2 project on GitHub.

@@ -517,5 +517,6 @@

Maintainers

+
diff --git a/spp_import_match/static/src/import_match_selector/import_match_selector.js b/spp_import_match/static/src/import_match_selector/import_match_selector.js new file mode 100644 index 00000000..78507446 --- /dev/null +++ b/spp_import_match/static/src/import_match_selector/import_match_selector.js @@ -0,0 +1,163 @@ +/** @odoo-module */ +import {ImportAction} from "@base_import/import_action/import_action"; +import {ImportDataSidepanel} from "@base_import/import_data_sidepanel/import_data_sidepanel"; +import {BaseImportModel} from "@base_import/import_model"; +import {patch} from "@web/core/utils/patch"; +import {_t} from "@web/core/l10n/translation"; + +patch(ImportAction.prototype, { + setup() { + super.setup(...arguments); + this.state.importMatchRecords = []; + this.state.selectedImportMatchId = false; + this.state.overwriteMatch = false; + }, + + async onWillStart() { + await super.onWillStart(...arguments); + await this._loadImportMatchRecords(); + }, + + async _loadImportMatchRecords() { + const records = await this.model.orm.searchRead( + "spp.import.match", + [["model_name", "=", this.resModel]], + ["id", "name", "overwrite_match"] + ); + this.state.importMatchRecords = records; + }, + + async onImportMatchChanged(matchId) { + this.state.selectedImportMatchId = matchId; + this.model.importMatchIds = matchId ? [matchId] : []; + if (matchId) { + const record = this.state.importMatchRecords.find((r) => r.id === matchId); + this.state.overwriteMatch = record ? record.overwrite_match : false; + } else { + this.state.overwriteMatch = false; + } + this.model.overwriteMatch = this.state.overwriteMatch; + }, + + onOverwriteMatchChanged(value) { + this.state.overwriteMatch = value; + this.model.overwriteMatch = value; + }, + + async handleImport(isTest) { + const res = await super.handleImport(...arguments); + if (!isTest && this.model.importMatchCounts) { + const counts = this.model.importMatchCounts; + const parts = []; + if (counts.created) { + parts.push(`${counts.created} ${_t("created")}`); + } + if (counts.overwritten) { + parts.push(`${counts.overwritten} ${_t("overwritten")}`); + } + if (counts.skipped) { + parts.push(`${counts.skipped} ${_t("skipped")}`); + } + if (parts.length) { + this.notification.add(parts.join(", "), { + type: "success", + }); + } + this.model.importMatchCounts = null; + } + return res; + }, +}); + +patch(ImportDataSidepanel, { + props: { + ...ImportDataSidepanel.props, + importMatchRecords: {type: Array, optional: true}, + selectedImportMatchId: {optional: true}, + onImportMatchChanged: {type: Function, optional: true}, + overwriteMatch: {type: Boolean, optional: true}, + onOverwriteMatchChanged: {type: Function, optional: true}, + }, +}); + +patch(ImportDataSidepanel.prototype, { + onImportMatchChange(ev) { + const matchId = parseInt(ev.target.value, 10) || false; + if (this.props.onImportMatchChanged) { + this.props.onImportMatchChanged(matchId); + } + }, + + onOverwriteMatchChange(ev) { + if (this.props.onOverwriteMatchChanged) { + this.props.onOverwriteMatchChanged(ev.target.checked); + } + }, +}); + +patch(BaseImportModel.prototype, { + async _callImport(dryrun, args) { + if (this.importMatchIds && this.importMatchIds.length) { + args[3] = { + ...args[3], + import_match_ids: this.importMatchIds, + overwrite_match: this.overwriteMatch || false, + }; + } + + try { + const res = await this.orm.silent.call( + "base_import.import", + "execute_import", + args, + { + dryrun, + context: { + ...this.context, + tracking_disable: this.importOptions.tracking_disable, + }, + } + ); + if (res.async === true) { + this.displayNotification(_t("Successfully added on Queue")); + history.go(-1); + } + if (res.import_match_counts) { + if (dryrun) { + const counts = res.import_match_counts; + const parts = []; + if (counts.created) { + parts.push(`${counts.created} ${_t("to create")}`); + } + if (counts.overwritten) { + parts.push(`${counts.overwritten} ${_t("to overwrite")}`); + } + if (counts.skipped) { + parts.push(`${counts.skipped} ${_t("to skip")}`); + } + if (parts.length) { + this._addMessage("info", [parts.join(", ")]); + } + } else { + this.importMatchCounts = res.import_match_counts; + } + } + return res; + } catch (error) { + return {error}; + } + }, + + displayNotification(message) { + this.env.services.action.doAction({ + type: "ir.actions.client", + tag: "display_notification", + params: { + title: "Queued", + message: message, + type: "success", + sticky: false, + }, + }); + }, +}); diff --git a/spp_import_match/static/src/import_match_selector/import_match_selector.xml b/spp_import_match/static/src/import_match_selector/import_match_selector.xml new file mode 100644 index 00000000..5eadd4fa --- /dev/null +++ b/spp_import_match/static/src/import_match_selector/import_match_selector.xml @@ -0,0 +1,73 @@ + + + + + + state.importMatchRecords or [] + state.selectedImportMatchId + onImportMatchChanged + state.overwriteMatch + onOverwriteMatchChanged + + + + + +
+

Import Matching

+ +
+ + +
+ Matched records will be overwritten with imported data. + Matched records will be skipped. +
+
+
+
+
+ +
diff --git a/spp_import_match/static/src/legacy/custom_base_import.xml b/spp_import_match/static/src/legacy/custom_base_import.xml deleted file mode 100644 index 45653cad..00000000 --- a/spp_import_match/static/src/legacy/custom_base_import.xml +++ /dev/null @@ -1,16 +0,0 @@ - - diff --git a/spp_import_match/static/src/legacy/js/custom_base_import.js b/spp_import_match/static/src/legacy/js/custom_base_import.js deleted file mode 100644 index 01ab3a0a..00000000 --- a/spp_import_match/static/src/legacy/js/custom_base_import.js +++ /dev/null @@ -1,54 +0,0 @@ -/** @odoo-module */ -import {BaseImportModel} from "@base_import/import_model"; -import {patch} from "@web/core/utils/patch"; -import {_t} from "@web/core/l10n/translation"; - -patch(BaseImportModel.prototype, { - setup() { - super.setup(); - }, - - async _callImport(dryrun, args) { - try { - const res = await this.orm.silent.call( - "base_import.import", - "execute_import", - args, - { - dryrun, - context: { - ...this.context, - tracking_disable: this.importOptions.tracking_disable, - }, - } - ); - if ("async" in res) { - if (res.async === true) { - this.displayNotification(_t("Successfully added on Queue")); - history.go(-1); - } - } - console.log(res); - return res; - } catch (error) { - // This pattern isn't optimal but it is need to have - // similar behaviours as in legacy. That is, catching - // all import errors and showing them inside the top - // "messages" area. - return {error}; - } - }, - - displayNotification(message) { - this.env.services.action.doAction({ - type: "ir.actions.client", - tag: "display_notification", - params: { - title: "Queued", - message: message, - type: "success", - sticky: false, - }, - }); - }, -}); diff --git a/spp_import_match/static/src/legacy/xml/custom_base_import.xml b/spp_import_match/static/src/legacy/xml/custom_base_import.xml deleted file mode 100644 index e1fdcfd2..00000000 --- a/spp_import_match/static/src/legacy/xml/custom_base_import.xml +++ /dev/null @@ -1,21 +0,0 @@ - - diff --git a/spp_import_match/tests/__init__.py b/spp_import_match/tests/__init__.py index 950a7d8a..ac9517c3 100644 --- a/spp_import_match/tests/__init__.py +++ b/spp_import_match/tests/__init__.py @@ -2,3 +2,5 @@ from . import test_import_match_model from . import test_base_write from . import test_queue_job +from . import test_base_load +from . import test_base_import_methods diff --git a/spp_import_match/tests/test_base_import_methods.py b/spp_import_match/tests/test_base_import_methods.py new file mode 100644 index 00000000..748e2cfb --- /dev/null +++ b/spp_import_match/tests/test_base_import_methods.py @@ -0,0 +1,163 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. + +from unittest.mock import patch + +from odoo.tests import TransactionCase, tagged + +from odoo.addons.queue_job.exception import FailedJobError + +from ..models.base_import import ImportValidationError + +OPTIONS = { + "import_skip_records": [], + "import_set_empty_fields": [], + "fallback_values": {}, + "name_create_enabled_fields": {}, + "encoding": "utf-8", + "separator": ",", + "quoting": '"', + "date_format": "", + "datetime_format": "", + "float_thousand_separator": ",", + "float_decimal_separator": ".", + "advanced": True, + "has_headers": True, + "keep_matches": False, + "limit": 2000, + "sheets": [], + "sheet": "", + "skip": 0, + "tracking_disable": True, +} + + +@tagged("post_install", "-at_install") +class TestBaseImportMethods(TransactionCase): + """Test SPPBaseImport helper methods.""" + + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.csv_options = { + "separator": ",", + "quoting": '"', + "encoding": "utf-8", + } + cls.import_record = cls.env["base_import.import"].create( + { + "res_model": "res.partner", + "file_name": "test.csv", + } + ) + + def test_csv_attachment_roundtrip(self): + """_create_csv_attachment + _read_csv_attachment round-trip.""" + fields_in = ["name", "email"] + data_in = [["Alice", "alice@test.com"], ["Bob", "bob@test.com"]] + attachment = self.import_record._create_csv_attachment(fields_in, data_in, self.csv_options, "roundtrip.csv") + self.assertTrue(attachment.id) + self.assertEqual(attachment.name, "roundtrip.csv") + fields_out, data_out = self.import_record._read_csv_attachment(attachment, self.csv_options) + self.assertEqual(fields_out, fields_in) + self.assertEqual(len(data_out), 2) + self.assertEqual(data_out[0], ["Alice", "alice@test.com"]) + self.assertEqual(data_out[1], ["Bob", "bob@test.com"]) + + def test_csv_attachment_custom_options(self): + """_create/_read_csv_attachment with custom separator/quoting/encoding.""" + custom_options = { + "separator": ";", + "quoting": "'", + "encoding": "latin-1", + } + fields_in = ["name", "email"] + data_in = [["TestAccent", "accent@test.com"]] + attachment = self.import_record._create_csv_attachment(fields_in, data_in, custom_options, "custom.csv") + fields_out, data_out = self.import_record._read_csv_attachment(attachment, custom_options) + self.assertEqual(fields_out, fields_in) + self.assertEqual(data_out[0][0], "TestAccent") + + def test_extract_chunks_basic(self): + """_extract_chunks splits data into chunks >= chunk_size.""" + model_obj = self.env["res.partner"] + fields = ["name", "email"] + data = [[f"P{i}", f"p{i}@test.com"] for i in range(7)] + chunks = list(self.env["base_import.import"]._extract_chunks(model_obj, fields, data, 3)) + self.assertTrue(len(chunks) > 1) + # All rows should be covered + all_rows = set() + for start, end in chunks: + for r in range(start, end + 1): + all_rows.add(r) + self.assertEqual(all_rows, set(range(7))) + + def test_extract_chunks_small_data(self): + """_extract_chunks with data smaller than chunk_size yields one chunk.""" + model_obj = self.env["res.partner"] + fields = ["name", "email"] + data = [["A", "a@test.com"], ["B", "b@test.com"]] + chunks = list(self.env["base_import.import"]._extract_chunks(model_obj, fields, data, 100)) + self.assertEqual(len(chunks), 1) + self.assertEqual(chunks[0], (0, 1)) + + def test_import_one_chunk_success(self): + """_import_one_chunk loads data successfully.""" + attachment = self.import_record._create_csv_attachment( + ["name"], [["ChunkSuccess99xyz"]], self.csv_options, "chunk.csv" + ) + result = self.import_record._import_one_chunk("res.partner", attachment, self.csv_options, {}) + errors = [m for m in result["messages"] if m.get("type") == "error"] + self.assertFalse(errors) + partner = self.env["res.partner"].search([("name", "=", "ChunkSuccess99xyz")]) + self.assertEqual(len(partner), 1) + + def test_import_one_chunk_error(self): + """_import_one_chunk raises FailedJobError on load errors.""" + attachment = self.import_record._create_csv_attachment(["name"], [["ErrTest"]], self.csv_options, "err.csv") + error_result = { + "messages": [{"type": "error", "message": "Test error"}], + "ids": [], + } + with patch.object(type(self.env["res.partner"]), "load", return_value=error_result): + with self.assertRaises(FailedJobError): + self.import_record._import_one_chunk("res.partner", attachment, self.csv_options, {}) + + def test_import_validation_error_attributes(self): + """ImportValidationError stores field, type, and message attrs.""" + err = ImportValidationError("Bad value", field="name", error_type="warning", field_type="char") + self.assertEqual(str(err), "Bad value") + self.assertEqual(err.message, "Bad value") + self.assertEqual(err.type, "warning") + self.assertEqual(err.field_path, ["name"]) + self.assertEqual(err.field_type, "char") + self.assertFalse(err.record) + self.assertTrue(err.not_matching_error) + # Test defaults + err2 = ImportValidationError("Default error") + self.assertEqual(err2.type, "error") + self.assertFalse(err2.field_path) + self.assertIsNone(err2.field_type) + + def test_execute_import_with_match_ids_passes_context(self): + """execute_import passes import_match_ids and overwrite_match to context.""" + res_partner_model = self.env["ir.model"].search([("model", "=", "res.partner")]) + name_field = self.env["ir.model.fields"].search( + [("name", "=", "name"), ("model_id", "=", res_partner_model.id)] + ) + self.env["res.partner"].create({"name": "ExecMatchTest99xyz", "email": "exec@test.com"}) + match = self.env["spp.import.match"].create({"model_id": res_partner_model.id, "overwrite_match": True}) + self.env["spp.import.match.fields"].create({"field_id": name_field.id, "match_id": match.id}) + import_rec = self.env["base_import.import"].create( + { + "res_model": "res.partner", + "file": b"name,email\nExecMatchTest99xyz,updated@test.com", + "file_name": "test_exec.csv", + "file_type": "csv", + } + ) + options = dict(OPTIONS) + options["import_match_ids"] = [match.id] + options["overwrite_match"] = True + result = import_rec.execute_import(["name", "email"], [], options, dryrun=True) + self.assertIn("import_match_counts", result) + self.assertEqual(result["import_match_counts"]["overwritten"], 1) diff --git a/spp_import_match/tests/test_base_load.py b/spp_import_match/tests/test_base_load.py new file mode 100644 index 00000000..74c17149 --- /dev/null +++ b/spp_import_match/tests/test_base_load.py @@ -0,0 +1,144 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. + +from odoo.tests import TransactionCase, tagged + +from ..models.base import _import_match_local + + +@tagged("post_install", "-at_install") +class TestBaseLoad(TransactionCase): + """Test the Base.load() override with import matching.""" + + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.res_partner_model = cls.env["ir.model"].search([("model", "=", "res.partner")]) + cls.name_field = cls.env["ir.model.fields"].search( + [("name", "=", "name"), ("model_id", "=", cls.res_partner_model.id)] + ) + cls.email_field = cls.env["ir.model.fields"].search( + [("name", "=", "email"), ("model_id", "=", cls.res_partner_model.id)] + ) + + def _create_match_rule(self, field_ids_data, overwrite=True): + """Helper to create a match rule with fields.""" + match = self.env["spp.import.match"].create( + { + "model_id": self.res_partner_model.id, + "overwrite_match": overwrite, + } + ) + for data in field_ids_data: + data["match_id"] = match.id + self.env["spp.import.match.fields"].create(data) + return match + + def test_load_no_usable_rules(self): + """When no usable rules exist, load() passes through to super().""" + result = self.env["res.partner"].load( + ["name", "email"], + [["NoRuleTest99xyz", "norule@test.com"]], + ) + self.assertFalse(result["messages"]) + partner = self.env["res.partner"].search([("name", "=", "NoRuleTest99xyz")]) + self.assertEqual(len(partner), 1) + + def test_load_match_overwrite_true(self): + """Match found + overwrite=True -> record updated.""" + partner = self.env["res.partner"].create({"name": "OverwriteTrue99xyz", "email": "old@test.com"}) + self._create_match_rule([{"field_id": self.name_field.id}], overwrite=True) + result = self.env["res.partner"].load( + ["name", "email"], + [["OverwriteTrue99xyz", "new@test.com"]], + ) + self.assertFalse(result["messages"]) + partner.invalidate_recordset() + self.assertEqual(partner.email, "new@test.com") + + def test_load_match_overwrite_false(self): + """Match found + overwrite=False -> record skipped.""" + partner = self.env["res.partner"].create({"name": "OverwriteFalse99xyz", "email": "original@test.com"}) + self._create_match_rule([{"field_id": self.name_field.id}], overwrite=False) + self.env["res.partner"].load( + ["name", "email"], + [["OverwriteFalse99xyz", "changed@test.com"]], + ) + partner.invalidate_recordset() + self.assertEqual(partner.email, "original@test.com") + + def test_load_no_match_creates_record(self): + """No match found -> new record created.""" + self._create_match_rule([{"field_id": self.name_field.id}]) + result = self.env["res.partner"].load( + ["name", "email"], + [["BrandNewPartner99xyz", "brandnew@test.com"]], + ) + self.assertFalse(result["messages"]) + partner = self.env["res.partner"].search([("name", "=", "BrandNewPartner99xyz")]) + self.assertEqual(len(partner), 1) + self.assertEqual(partner.email, "brandnew@test.com") + + def test_load_multiple_records_mixed(self): + """Mix of matched and new records in one import.""" + self.env["res.partner"].create({"name": "ExistingMixed99xyz", "email": "existing@test.com"}) + self._create_match_rule([{"field_id": self.name_field.id}], overwrite=True) + result = self.env["res.partner"].load( + ["name", "email"], + [ + ["ExistingMixed99xyz", "updated@test.com"], + ["NewMixed99xyz", "newmixed@test.com"], + ], + ) + self.assertFalse(result["messages"]) + existing = self.env["res.partner"].search([("name", "=", "ExistingMixed99xyz")]) + self.assertEqual(len(existing), 1) + existing.invalidate_recordset() + self.assertEqual(existing.email, "updated@test.com") + new_partner = self.env["res.partner"].search([("name", "=", "NewMixed99xyz")]) + self.assertEqual(len(new_partner), 1) + + def test_load_with_import_match_ids_context(self): + """Context import_match_ids filters to specific rules.""" + self._create_match_rule([{"field_id": self.name_field.id}]) + match2 = self._create_match_rule([{"field_id": self.email_field.id}]) + partner = self.env["res.partner"].create({"name": "CtxFilter99xyz", "email": "ctxfilter@test.com"}) + # Only use match2 (email rule); import with matching email but different name + result = ( + self.env["res.partner"] + .with_context(import_match_ids=[match2.id]) + .load( + ["name", "email"], + [["CtxFilterRenamed99", "ctxfilter@test.com"]], + ) + ) + self.assertFalse(result["messages"]) + partner.invalidate_recordset() + self.assertEqual(partner.name, "CtxFilterRenamed99") + + def test_load_appends_id_field(self): + """Auto-appends 'id' when not in fields list.""" + self._create_match_rule([{"field_id": self.name_field.id}]) + fields = ["name", "email"] + self.env["res.partner"].load( + fields, + [["AppendIdTest99xyz", "appendid@test.com"]], + ) + self.assertIn("id", fields) + + def test_load_match_counts_tracking(self): + """_import_match_local.counts populated when import_match_ids in context.""" + self.env["res.partner"].create({"name": "CountsExisting99xyz", "email": "counts@test.com"}) + match = self._create_match_rule([{"field_id": self.name_field.id}], overwrite=True) + _import_match_local.counts = None + self.env["res.partner"].with_context(import_match_ids=[match.id]).load( + ["name", "email"], + [ + ["CountsExisting99xyz", "updated@test.com"], + ["CountsNew99xyz", "new@test.com"], + ], + ) + counts = _import_match_local.counts + self.assertIsNotNone(counts) + self.assertEqual(counts["overwritten"], 1) + self.assertEqual(counts["created"], 1) + self.assertEqual(counts["skipped"], 0) diff --git a/spp_import_match/tests/test_import_match_model.py b/spp_import_match/tests/test_import_match_model.py index 5c236b57..bc82b80e 100644 --- a/spp_import_match/tests/test_import_match_model.py +++ b/spp_import_match/tests/test_import_match_model.py @@ -162,3 +162,68 @@ def test_field_compute_name_with_sub_field(self): ] ) self.assertEqual(match.field_ids[0].name, "child_ids/name") + + def test_onchange_field_id_skips_relational(self): + """_onchange_field_id skips duplicate check for relational fields.""" + parent_id_field = self.env["ir.model.fields"].search( + [("name", "=", "parent_id"), ("model_id", "=", self.res_partner_model.id)], + limit=1, + ) + match = self._create_match_rule( + [ + {"field_id": parent_id_field.id}, + {"field_id": parent_id_field.id}, + ] + ) + # Should NOT raise ValidationError because parent_id is many2one + for field_rec in match.field_ids: + field_rec._onchange_field_id() + self.assertEqual(len(match.field_ids), 2) + + def test_match_find_sub_field(self): + """_match_find handles sub_field matching for relational fields.""" + child = self.env["res.partner"].create({"name": "SubFieldChild_Uniq99xyz"}) + parent = self.env["res.partner"].create( + { + "name": "SubFieldParent_Uniq99xyz", + "child_ids": [(4, child.id)], + } + ) + child_ids_field = self.env["ir.model.fields"].search( + [ + ("name", "=", "child_ids"), + ("model_id", "=", self.res_partner_model.id), + ], + limit=1, + ) + match = self._create_match_rule( + [ + { + "field_id": child_ids_field.id, + "sub_field_id": self.name_field.id, + } + ] + ) + result = match._match_find( + self.env["res.partner"], + {"child_ids": [(0, 0, {"name": "SubFieldChild_Uniq99xyz"})]}, + {"child_ids/name": "SubFieldChild_Uniq99xyz", "id": None}, + ) + self.assertEqual(result, parent) + + def test_match_find_multiple_combinations(self): + """_match_find iterates rules; first matching single result wins.""" + partner = self.env["res.partner"].create({"name": "MultiCombo99xyz", "email": "multicombo@test.com"}) + # First rule by email (lower sequence -> tried first) + match1 = self._create_match_rule([{"field_id": self.email_field.id}]) + match1.sequence = 1 + # Second rule by name + match2 = self._create_match_rule([{"field_id": self.name_field.id}]) + match2.sequence = 2 + # Email won't match, but name will + result = self.env["spp.import.match"]._match_find( + self.env["res.partner"], + {"name": "MultiCombo99xyz", "email": "nomatch@test.com"}, + {"name": "MultiCombo99xyz", "email": "nomatch@test.com", "id": None}, + ) + self.assertEqual(result, partner) diff --git a/spp_oauth/README.rst b/spp_oauth/README.rst index 7d681a1b..d5157bd7 100644 --- a/spp_oauth/README.rst +++ b/spp_oauth/README.rst @@ -1,18 +1,22 @@ +.. image:: https://odoo-community.org/readme-banner-image + :target: https://odoo-community.org/get-involved?utm_source=readme + :alt: Odoo Community Association + ================== OpenSPP API: Oauth ================== -.. +.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !! This file is generated by oca-gen-addon-readme !! !! changes will be overwritten. !! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! - !! source digest: sha256:25a909cff59dac7bf6fb0a36671aac5a2b03bbd13eda1b5085bde0c2f467c93f + !! source digest: sha256:a65143c0c6e2b781e1b4ed36d9f6c8cd8376811fdc934ebd3373f1b019cb0afa !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -.. |badge1| image:: https://img.shields.io/badge/maturity-Alpha-red.png +.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png :target: https://odoo-community.org/page/development-status - :alt: Alpha + :alt: Beta .. |badge2| image:: https://img.shields.io/badge/license-LGPL--3-blue.png :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html :alt: License: LGPL-3 @@ -43,12 +47,12 @@ Key Capabilities Key Models ~~~~~~~~~~ -+-------------------------+--------------------------------------------+ -| Model | Description | -+=========================+============================================+ -| ``res.config.settings`` | Extended to add OAuth private and public | -| | key fields | -+-------------------------+--------------------------------------------+ ++-------------------------+-------------------------------------------+ +| Model | Description | ++=========================+===========================================+ +| ``res.config.settings`` | Extended to add OAuth private and public | +| | key fields | ++-------------------------+-------------------------------------------+ Utility Functions ~~~~~~~~~~~~~~~~~ @@ -118,10 +122,6 @@ Dependencies **External Python**: ``pyjwt>=2.4.0`` -.. IMPORTANT:: - This is an alpha version, the data model and design can change at any time without warning. - Only for development or testing purpose, do not use in production. - **Table of contents** .. contents:: @@ -164,4 +164,4 @@ Current maintainers: This module is part of the `OpenSPP/OpenSPP2 `_ project on GitHub. -You are welcome to contribute. \ No newline at end of file +You are welcome to contribute. diff --git a/spp_oauth/__manifest__.py b/spp_oauth/__manifest__.py index 2468bbe2..3dcfe69b 100644 --- a/spp_oauth/__manifest__.py +++ b/spp_oauth/__manifest__.py @@ -5,9 +5,9 @@ "category": "OpenSPP", "version": "19.0.1.3.1", "author": "OpenSPP.org", - "development_status": "Alpha", + "development_status": "Beta", "maintainers": ["jeremi", "gonzalesedwin1123", "reichie020212"], - "external_dependencies": {"python": ["pyjwt>=2.4.0"]}, + "external_dependencies": {"python": ["pyjwt>=2.4.0", "cryptography"]}, "website": "https://github.com/OpenSPP/OpenSPP2", "license": "LGPL-3", "depends": [ diff --git a/spp_oauth/static/description/index.html b/spp_oauth/static/description/index.html index 372b609b..622eaaa9 100644 --- a/spp_oauth/static/description/index.html +++ b/spp_oauth/static/description/index.html @@ -3,7 +3,7 @@ -OpenSPP API: Oauth +README.rst -
-

OpenSPP API: Oauth

+
+ + +Odoo Community Association + +
+

OpenSPP API: Oauth

-

Alpha License: LGPL-3 OpenSPP/OpenSPP2

+

Beta License: LGPL-3 OpenSPP/OpenSPP2

OAuth 2.0 authentication framework for securing OpenSPP API communications using JWT tokens signed with RSA keys. Provides utility functions to generate and verify JWT signatures using the RS256 algorithm. Stores RSA key pairs as system parameters and exposes configuration UI for key management.

-

Key Capabilities

+

Key Capabilities

  • Generate JWT tokens signed with RSA private keys using calculate_signature()
  • @@ -389,7 +394,7 @@

    Key Capabilities

-

Key Models

+

Key Models

@@ -409,7 +414,7 @@

Key Models

-

Utility Functions

+

Utility Functions

@@ -437,7 +442,7 @@

Utility Functions

-

Configuration

+

Configuration

After installing:

  1. Navigate to Settings > General Settings
  2. @@ -454,7 +459,7 @@

    Configuration

-

UI Location

+

UI Location

  • Settings App Block: SPP OAuth Settings (within Settings > General Settings)
  • @@ -462,7 +467,7 @@

    UI Location

-

Security

+

Security

@@ -483,7 +488,7 @@

Security

in ir.config_parameter.

-

Extension Points

+

Extension Points

  • Import calculate_signature() and verify_and_decode_signature() from odoo.addons.spp_oauth.tools to implement OAuth 2.0 @@ -493,14 +498,9 @@

    Extension Points

-

Dependencies

+

Dependencies

spp_security, base

External Python: pyjwt>=2.4.0

-
-

Important

-

This is an alpha version, the data model and design can change at any time without warning. -Only for development or testing purpose, do not use in production.

-

Table of contents

    @@ -513,7 +513,7 @@

    Dependencies

-

Bug Tracker

+

Bug Tracker

Bugs are tracked on GitHub Issues. In case of trouble, please check there if your issue has already been reported. If you spotted it first, help us to smash it by providing a detailed and welcomed @@ -521,15 +521,15 @@

Bug Tracker

Do not contact contributors directly about support or help with technical issues.

-

Credits

+

Credits

-

Authors

+

Authors

  • OpenSPP.org
-

Maintainers

+

Maintainers

Current maintainers:

jeremi gonzalesedwin1123 reichie020212

This module is part of the OpenSPP/OpenSPP2 project on GitHub.

@@ -538,5 +538,6 @@

Maintainers

+ diff --git a/spp_oauth/tools/rsa_encode_decode.py b/spp_oauth/tools/rsa_encode_decode.py index 25dbd70d..af746649 100644 --- a/spp_oauth/tools/rsa_encode_decode.py +++ b/spp_oauth/tools/rsa_encode_decode.py @@ -61,5 +61,5 @@ def verify_and_decode_signature(env, access_token): pubkey = get_public_key(env) try: return jwt.decode(access_token, key=pubkey, algorithms=[JWT_ALGORITHM]) - except Exception as e: + except jwt.exceptions.PyJWTError as e: raise OpenSPPOAuthJWTException(str(e)) from e diff --git a/spp_oauth/views/res_config_view.xml b/spp_oauth/views/res_config_view.xml index cdbda30b..ca750153 100644 --- a/spp_oauth/views/res_config_view.xml +++ b/spp_oauth/views/res_config_view.xml @@ -10,7 +10,7 @@