Remodela institutições (versão 3) - wip #1263

robertatakenaka · 2026-01-30T22:52:45Z

O que esse PR faz?

Fale sobre o propósito do pull request como por exemplo: quais problemas ele soluciona ou quais features ele adiciona.

Onde a revisão poderia começar?

Indique o caminho do arquivo e o arquivo onde o revisor deve iniciar a leitura do código.

Como este poderia ser testado manualmente?

Estabeleça os passos necessários para que a funcionalidade seja testada manualmente pelo revisor.

Algum cenário de contexto que queira dar?

Indique um contexto onde as modificações se fazem necessárias ou passe informações que contextualizam
o revisor a fim de facilitar o entendimento da funcionalidade.

Screenshots

Quando aplicável e se fizer possível adicione screenshots que remetem a situação gráfica do problema que o pull request resolve.

Quais são tickets relevantes?

Indique uma issue ao qual o pull request faz relacionamento.

Referências

Indique as referências utilizadas para a elaboração do pull request.

… na v1

… metadados de fonte

…nizationalLevel

… de organization

Copilot

Pull request overview

This PR refactors the organization/institution models from version 2 to version 3, introducing new models and restructuring the existing ones. The title indicates this is work in progress ("wip").

Changes:

Introduces new models: OrganizationalLevel, RawOrganization, and BaseOrganizationalLevel
Refactors Organization model to remove institution_type_mec, institution_type_scielo, and is_official fields
Adds new fields: source, external_id to Organization and OrganizationInstitutionType
Adds new utility function clean_xml_tag_content to standardize text input
Updates institution models with documentation about their legacy status (version 1)

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 21 comments.

Show a summary per file

File	Description
organization/models.py	Major refactoring of Organization models, adding RawOrganization and OrganizationalLevel, removing old fields and methods
organization/migrations/0011_organizationallevel_raworganization_and_more.py	Django migration creating new models and altering existing ones
organization/choices.py	Replaces inst_type tuple with SOURCE_CHOICES list
institution/models.py	Adds documentation marking these models as version 1 (legacy)
core/utils/standardizer.py	Adds clean_xml_tag_content and remove_html_tags functions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-30T22:58:58Z

organization/migrations/0011_organizationallevel_raworganization_and_more.py

+            field=models.ManyToManyField(
+                blank=True,
+                help_text="Standardized levels (requires organization)",
+                related_name="raw_organization_leves",


Typo in related_name: "raw_organization_leves" should be "raw_organization_levels" (missing 'l'). This should match the corrected name in the model.

Suggested change

related_name="raw_organization_leves",

related_name="raw_organization_levels",

Copilot · 2026-01-30T22:58:58Z

organization/models.py

+    class Meta:
+        verbose_name = _("Organization")
+        verbose_name_plural = _("Organizations")
+        unique_together = [("name", "location", "external_id", "source")]  # NOVO


The unique_together constraint includes 'external_id' and 'source' which are nullable fields. This means multiple records with NULL values for these fields can exist, which may not provide the intended uniqueness. If the intent is to allow multiple organizations with the same name in the same location but different sources/external_ids, consider adding a database constraint that handles NULLs appropriately, or ensure that source always has a value (remove null=True).

Copilot · 2026-01-30T22:58:59Z

organization/models.py

+    def get(cls, name, original=None, normalized_name=None, acronym=None, country=None, state=None, city=None, level_1=None, level_2=None, level_3=None):
+        """
+        Busca registro existente.
+        """
+        if not name:
+            raise ValueError("RawOrganization.get requires name")
+
+        original = clean_xml_tag_content(original)
+        normalized_name = clean_xml_tag_content(normalized_name)
+        name = clean_xml_tag_content(name)
+        country = clean_xml_tag_content(country)
+        state = clean_xml_tag_content(state)
+        city = clean_xml_tag_content(city)
+        level_1 = clean_xml_tag_content(level_1)
+        level_2 = clean_xml_tag_content(level_2)
+        level_3 = clean_xml_tag_content(level_3)
+
+        params = {"name__iexact": name}
+
+        if original:
+            params["original__iexact"] = original
+        else:
+            params["original__isnull"] = True
+
+        if normalized_name:
+            params["normalized_name__iexact"] = normalized_name
+        else:
+            params["normalized_name__isnull"] = True
+
+        if country:
+            params["country__iexact"] = country
+        else:
+            params["country__isnull"] = True
+
+        if state:
+            params["state__iexact"] = state
+        else:
+            params["state__isnull"] = True
+
+        if city:
+            params["city__iexact"] = city
+        else:
+            params["city__isnull"] = True
+
+        if level_1:
+            params["level_1__iexact"] = level_1
+        else:
+            params["level_1__isnull"] = True
+
+        if level_2:
+            params["level_2__iexact"] = level_2
+        else:
+            params["level_2__isnull"] = True
+
+        if level_3:
+            params["level_3__iexact"] = level_3
+        else:
+            params["level_3__isnull"] = True
+
+        try:
+            return cls.objects.get(**params)
+        except cls.MultipleObjectsReturned:
+            return cls.objects.filter(**params).first()


The get method builds query parameters checking for None values and setting __isnull conditions. However, the method doesn't handle the case where 'acronym' is provided but is None in the parameters - line 844 shows acronym is accepted but never checked in the params dictionary construction. This inconsistency could lead to unexpected query behavior. Consider adding similar handling for acronym as done for other optional fields.

Copilot · 2026-01-30T22:58:59Z

organization/models.py

+            if organization is not None:
+                obj.organization = organization
+            if organizational_level is not None:
+                obj.organizational_level = organizational_level


Cannot assign ManyToMany fields directly. Lines 1027, 1029 attempt to assign values directly to ManyToMany fields 'organization' and 'organizational_level'. This will raise a TypeError. Use .set() or .add() methods instead, and these operations can only be performed after the object is saved.

Copilot · 2026-01-30T22:58:59Z

core/utils/standardizer.py

+        if text_.isalpha():
+            return text_
+        else:
+            return None


The logic in clean_xml_tag_content is problematic. When assert_string=True (which is the default), it uses text_.isalpha() to validate the string. This method returns True only if ALL characters are alphabetic (no spaces, numbers, or special characters). This means that most organization names like "Universidade de São Paulo", "USP-2024", or any name with spaces will be rejected and return None. This will cause data loss when cleaning organization names that contain spaces, numbers, or special characters.

Suggested change

if text_.isalpha():

return text_

else:

return None

# Accept strings that contain at least one alphabetic character.

# This allows spaces, digits, and punctuation in names (e.g., "USP-2024"),

# while still rejecting strings with no letters at all.

if any(ch.isalpha() for ch in text_):

return text_

return None

Copilot · 2026-01-30T22:59:02Z

organization/models.py

+    organizational_level = models.ManyToManyField(
+        "OrganizationalLevel",
+        blank=True,
+        related_name="raw_organization_leves",


Typo in related_name: "raw_organization_leves" should be "raw_organization_levels" (missing 'l').

Suggested change

related_name="raw_organization_leves",

related_name="raw_organization_levels",

Copilot · 2026-01-30T22:59:02Z

organization/models.py

+    @property
+    def is_matched(self):
+        """Indica se está vinculado a Organization."""
+        return self.organization_id is not None


The is_matched property checks 'self.organization_id' but 'organization' is defined as a ManyToManyField, not a ForeignKey. ManyToMany fields don't have an '_id' attribute. This will raise an AttributeError. To check if a ManyToMany relationship exists, use 'self.organization.exists()' instead (though this requires the object to be saved first).

Suggested change

return self.organization_id is not None

return self.organization.exists()

Copilot · 2026-01-30T22:59:02Z

organization/models.py

+                organization=organization,
+                organizational_level=organizational_level,
+                match_status=match_status or "unmatched",
+                extra_data=extra_data,


The create method references 'extra_data' field (line 949) which is not defined in the RawOrganization model. This will raise a TypeError when trying to create an instance with this parameter.

Suggested change

extra_data=extra_data,

Copilot · 2026-01-30T22:59:03Z

organization/models.py

+        self.organization = organization
+        self.organizational_level = organizational_level


Cannot assign ManyToMany fields directly. Lines 1072-1073 attempt to assign values directly to ManyToMany fields 'organization' and 'organizational_level'. This will raise a TypeError. These should use .set() or .add() methods: e.g., 'self.organization.set([organization])' if organization is a single instance, or 'self.organization.add(organization)'.

Copilot · 2026-01-30T22:59:03Z

organization/models.py

 from core.forms import CoreAdminModelForm
 from core.models import BaseHistory, CommonControlField
-from core.utils.standardizer import remove_extra_spaces
+from core.utils.standardizer import clean_xml_tag_content, remove_extra_spaces


Import of 'remove_extra_spaces' is not used.

Suggested change

from core.utils.standardizer import clean_xml_tag_content, remove_extra_spaces

from core.utils.standardizer import clean_xml_tag_content

robertatakenaka added 8 commits January 30, 2026 14:59

docs(institution): adicionar docstrings de legado e mapeamento de uso…

0257877

… na v1

refactor(organization): remover inst_type redundante em choices.py

705e99e

feat(organization): refatorar modelos v2, adicionar status de dados e…

eb53cf4

… metadados de fonte

Cria definição de SOURCE_CHOICES e DATA_STATUS_CHOICES

5bb8309

refactor(organization): remove deprecated DATA_STATUS_CHOICES

b562cd3

feat(core): add HTML/XML tag cleaning utilities to standardizer

4d4c094

feat(organization): implement v3 models with RawOrganization and Orga…

c6a4845

…nizationalLevel

Corrige defeitos encontrados ao executar a migração e cria a migração…

43ee9a9

… de organization

Copilot AI review requested due to automatic review settings January 30, 2026 22:52

Copilot started reviewing on behalf of robertatakenaka January 30, 2026 22:53 View session

Copilot AI reviewed Jan 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remodela institutições (versão 3) - wip #1263

Remodela institutições (versão 3) - wip #1263

Uh oh!

robertatakenaka commented Jan 30, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Copilot AI Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	related_name="raw_organization_leves",
	related_name="raw_organization_levels",

	return self.organization_id is not None
	return self.organization.exists()

		self.organization = organization
		self.organizational_level = organizational_level

	from core.utils.standardizer import clean_xml_tag_content, remove_extra_spaces
	from core.utils.standardizer import clean_xml_tag_content

Remodela institutições (versão 3) - wip #1263

Are you sure you want to change the base?

Remodela institutições (versão 3) - wip #1263

Uh oh!

Conversation

robertatakenaka commented Jan 30, 2026

O que esse PR faz?

Onde a revisão poderia começar?

Como este poderia ser testado manualmente?

Algum cenário de contexto que queira dar?

Screenshots

Quais são tickets relevantes?

Referências

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant