[ADD] data cleaning: overview

closes odoo/documentation#10013

X-original-commit: 8ca247d9dd
Signed-off-by: Samuel Lieber (sali) <sali@odoo.com>
Co-authored-by: ksc-odoo <73958186+ksc-odoo@users.noreply.github.com>

content/applications/productivity.rst

@@ -13,6 +13,7 @@ Productivity
     productivity/knowledge
     productivity/calendar
     productivity/discuss
+    productivity/data_cleaning
     productivity/whatsapp
     productivity/voip
     productivity/to_do

content/applications/productivity/data_cleaning.rst

@@ -0,0 +1,234 @@
+=============
+Data Cleaning
+=============
+
+The Odoo *Data Cleaning* app keeps the database neat and organized throughout the use of its
+provided features:
+
+- :ref:`Deduplicates <data_cleaning/deduplication>`: merges or removes duplicate entries to ensure
+  data is unique.
+- :ref:`Recycles <data_cleaning/recycle>`: identifies outdated records to either archive or delete
+  them.
+- :ref:`Merges <data_cleaning/merge-action-manager>`: combines multiple similar entries into one
+  streamlined record.
+- :ref:`Formats <data_cleaning/field-cleaning>`: standardizes text data by finding and replacing it
+  according to specified needs.
+
+Through customizable automated rules (or manual praxis), individual records and accompanying text
+data in the database will remain up-to-date and consistently formatted, tailored to company
+specifications.
+
+.. _data_cleaning/install-modules:
+
+Install modules
+===============
+
+The *Data Cleaning* application consists of several modules. :ref:`Install <general/install>` the
+following to access all available features:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 25 50
+
+   * - Name
+     - Technical name
+     - Description
+   * - :guilabel:`Data Recycle`
+     - `data_recycle`
+     - Base module to enable the recycle feature, available on Odoo Community edition.
+   * - :guilabel:`Data Cleaning`
+     - `data_cleaning`
+     - Enables field cleaning feature to format text data across multiple records, available
+       **only** on Odoo Enterprise edition.
+   * - :guilabel:`Data Cleaning (merge)`
+     - `data_merge`
+     - Enables the deduplication feature to find similar (or duplicate) records, and merge them,
+       available **only** on Odoo Enterprise edition.
+
+.. spoiler:: Additionally, several app-specific modules are available
+
+   .. list-table::
+      :widths: 25 25 50
+
+      * - :guilabel:`CRM Deduplication`
+        - `data_merge_crm`
+        - Enables the deduplication feature on the *CRM* app, and uses the :doc:`CRM default merging
+          feature <../sales/crm/pipeline/merge_similar>`.
+      * - :guilabel:`Helpdesk Merge action`
+        - `data_merge_helpdesk`
+        - Enables the merge feature for the *Helpdesk* app.
+      * - :guilabel:`Project Merge action`
+        - `data_merge_project`
+        - Enables the merge feature for the *Projects* app.
+      * - :guilabel:`UTM Deduplication`
+        - `data_merge_utm`
+        - Enables the merge feature for the *UTM Tracker* app.
+      * - :guilabel:`WMS Accounting Merge`
+        - `data_merge_stock_account`
+        - Creates a warning in cases of products merging that may affect inventory valuation, if the
+          *Inventory* app is installed.
+
+.. _data_cleaning/deduplication:
+
+Deduplication
+=============
+
+On the :guilabel:`Duplicates` dashboard (:menuselection:`Data Cleaning app --> Deduplication`), Odoo
+suggests groups of similar records to be :ref:`merged <data_cleaning/merge-records>` by matching
+conditions within the records set by the :ref:`deduplication rules
+<data_cleaning/deduplication-rules>`.
+
+.. image:: data_cleaning/data-cleaning-duplicates.png
+   :align: center
+   :alt: Deduplication dashboard in the Data Cleaning application.
+
+The :guilabel:`RULE` sidebar lists each of the active deduplication rules, and displays the total
+number of duplicates detected beside each rule.
+
+By default, the :guilabel:`All` rule is selected. Displayed records are grouped by their rule, with
+a :guilabel:`Similarity` rating (out of 100%) in the list view, with the following columns:
+
+- :guilabel:`Created On`: the date and time the original record was created.
+- :guilabel:`Name`: the name or title of the original record.
+- :guilabel:`Field Values`: the original record's values for the fields used to detect duplicates.
+- :guilabel:`Used In`: lists other models referencing the original record.
+- :guilabel:`ID`: the original record's unique ID.
+- :guilabel:`Is Master`: the duplicates are merged into the *master* record. There can be **one**
+  master record in a grouping of similar records.
+
+Select a specific rule in the :guilabel:`RULE` sidebar to filter the duplicate records.
+
+.. _data_cleaning/merge-records:
+
+Merge duplicate records
+-----------------------
+
+To merge records, first choose a *master* record within the grouping of similar records. The master
+record acts as the base, at which any additional information from similar records are merged into.
+
+Optionally, no master record can be set, leaving Odoo to choose a record at random to merge into.
+
+Next, click the :guilabel:`Merge` button at the top of the similar records grouping. Then, click
+:guilabel:`Ok` to confirm the merge.
+
+Once a record is merged, a message is logged in the chatter of the master record, describing the
+merge. Certain records, like *Project* tasks, are logged in the chatter with a link to the old
+record for a convenient reference.
+
+.. tip::
+   Discard groupings by clicking the :guilabel:`DISCARD` button. Upon doing so, the grouping is
+   hidden from the list and archived.
+
+   View discarded groupings by selecting the :guilabel:`Discarded` filter from the :ref:`Search...
+   <search/filters>` bar.
+
+.. _data_cleaning/deduplication-rules:
+
+Deduplication rules
+-------------------
+
+The :guilabel:`Deduplication Rules` page (:menuselection:`Data Cleaning app --> Configuration -->
+Rules: Deduplication`) is where the conditions for records to be detected as duplicates can be set.
+
+These rules can be configured for each model in the database, and with varying levels of
+specificity.
+
+.. tip::
+   The deduplication rules run once every day, by default, as part of a scheduled action chron
+   (*Data Merge: Find Duplicate Records*). However, each rule can be :ref:`ran manually
+   <data_cleaning/run-deduplication-rule>` anytime.
+
+Modify a deduplication rule
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Select a default rule to edit, or create a new rule by clicking on the :guilabel:`New` button, on
+the :guilabel:`Deduplication Rules` page (:menuselection:`Data Cleaning app --> Configuration -->
+Rules: Deduplication`).
+
+First, choose a :guilabel:`Model` for this rule to target. Selecting a model updates the rule title
+to the chosen model.
+
+Optionally, configure a :guilabel:`Domain` to specify the records eligible for this rule. The number
+of eligible records is shown in the :icon:`oi-arrow-right` :guilabel:`# record(s)` link.
+
+Depending on the selected :guilabel:`Model`, the :guilabel:`Duplicate Removal` field appears.
+Choose whether to :guilabel:`Archive` or :guilabel:`Delete` merged records.
+
+Next, select a :guilabel:`Merge Mode`:
+
+- :guilabel:`Manual`: requires each duplicate grouping to be manually merged, also enables the
+  :guilabel:`Notify Users` field.
+- :guilabel:`Automatic`: automatically merges duplicate groupings, without notifying users, based on
+  the records with a similarity percentage above the threshold set in the :guilabel:`Similarity
+  Threshold` field.
+
+Enable the :guilabel:`Active` toggle to start capturing duplicates with this rule as soon as it is
+saved.
+
+Lastly, create at least one deduplication rule in the :guilabel:`Deduplication Rules` field, by
+clicking :guilabel:`Add a line`, under the :guilabel:`Unique ID Field` column.
+
+- Select a field in the model from the :guilabel:`Unique ID Field` drop-down menu. This field is
+  referenced for similar records.
+- Select a matching condition in the :guilabel:`Match If` field to apply the deduplication rule,
+  depending on the text in the :guilabel:`Unique ID Field`:
+
+  - :guilabel:`Exact Match`: the characters in the text match exactly.
+  - :guilabel:`Case/Accent Insensitive Match`: the characters in the text match, regardless of
+    casing and language-specific accent differences.
+
+.. important::
+   The rule does **not** capture duplicates without at least one deduplication rule set in the
+   :guilabel:`Deduplication Rules` field.
+
+.. tip::
+   A few more fields are available for an advanced configuration.
+
+   If on a multi-company database, the :guilabel:`Cross-Company` field is available. When enabled,
+   duplicates across different companies are suggested.
+
+   Activate :ref:`developer-mode` to display the :guilabel:`Suggestion Threshold` field. Duplicates
+   with a similarity below the threshold set in this field are **not** suggested.
+
+With the rule's configuration complete, either close the rule form, or :ref:`run the rule manually
+<data_cleaning/run-deduplication-rule>` to instantly capture duplicate records.
+
+.. _data_cleaning/run-deduplication-rule:
+
+Manually run a deduplication rule
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To manually run a specific deduplication rule at any time, navigate to :menuselection:`Data Cleaning
+app --> Configuration --> Rules: Deduplication`, and select the rule to run.
+
+Then, on the rule form, select the :guilabel:`Deduplicate` button on the top-left. Upon doing so,
+the :icon:`fa-clone` :guilabel:`Duplicates` smart button displays the number of duplicates captured.
+
+Click on the :icon:`fa-clone` :guilabel:`Duplicates` smart button to manage these records.
+
+.. _data_cleaning/recycle:
+
+Recycle records
+===============
+
+On the :guilabel:`Field Recycle Records` dashboard (:menuselection:`Data Cleaning app --> Recycle
+Records`), Odoo detects records that can be archived or deleted, by matching conditions within the
+records set by the recycle record's rules.
+
+.. _data_cleaning/field-cleaning:
+
+Field cleaning
+==============
+
+On the :guilabel:`Field Cleaning Records* dashboard (:menuselection:`Data Cleaning app --> Field
+Cleaning`), Odoo suggests formatting changes to data in fields of a record, to follow a convention
+set by the field cleaning rules.
+
+.. _data_cleaning/merge-action-manager:
+
+Merge action manager
+====================
+
+The :guilabel:`Merge Action Manager` (:menuselection:`Data Cleaning app --> Configuration --> Merge
+Action Manager`) enables or disables the *Merge* action available in the *Actions* menu for models
+in the database.