Nice Classification Recommendation using Sentence-BERT
The trademark protection trade-off
To obtain a trademark, a brand must be unique and clearly distinguish its Goods and Services from those of its competitors. Intellectual Property offices worldwide use a trademark classification system that groups similar products or services into 45 Classes. Goods are in Classes 1 to 34, and Services are in Classes 35 to 45. Each Class has a general heading that provides a broad explanation of what is included in the Class, an Explanatory Note describing what the Class includes and does not include in particular, and an Alphabetical List of specific products or services.
Properly identifying the Goods and Services that you intend to associate with any proposed trademark is of vital importance because the scope of protection of that trademark is only applicable to the specific international Classes that you designate and pay for in your application, and it is not possible to add Goods or Services after the application has been filed.
Since there is no limit to the number of Goods or Services that can be included in a single Class, it may be tempting to choose a broad range. However, this increases the likelihood that the trademark will be challenged by others or that it will be revoked for being too similar to an existing trademark or for not being used. It is important to carefully consider the Goods and Services included in the trademark application to avoid these potential issues.
The task of choosing the correct Classes and indications is further complicated by the fact there are currently close to 10,000 options defined by the Nice Classification System.
Sentence-BERT to the rescue
Sentence-BERT (SBERT) is a natural language processing model that was trained using the technique of Bidirectional Encoder Representations from Transformers (BERT). SBERT is designed to understand and process sentences to improve the performance of Natural Language Processing tasks such as text classification, sentiment analysis, and question answering. SBERT uses a combination of self-attention mechanisms and transformers to learn contextual representations of words in a sentence, which allows it to capture the meaning of a sentence as a whole.
Get Brice Hoareau’s stories in your inbox
Join Medium for free to get updates from this writer.
SBERT represents entire sentences, and their semantic information, as vectors. The vector of one sentence can be compared to the vector of another using metrics such as Cosine Similarity: the higher the value, the more similar the sentences are.
Implementation
In our use case, the 10k Nice headings, explanatory notes, and exact classifications are vectorised and the Cosine Similarity of each vector in the matrix and a vectorised product (or service) description are calculated. The classifications with the highest similarity are presented to the user.
#Load the necessary libraries
import pandas as pd
import numpy as np
from sentence_transformers import SentenceTransformer
#Define the Cosine Similarity function
def cosine(u, v):
return np.dot(u, v) / (np.linalg.norm(u) * np.linalg.norm(v))
#Load the Nice classification data
ncl_all = pd.read_csv('consolidated_nice_classifications.csv', sep=',')
#Load the SBERT model
sbert_model = SentenceTransformer('all-MiniLM-L6-v2')
#Create the sentence embeddings for the Nice classifications
sentence_embeddings = sbert_model.encode(ncl_all['Desc'])
#Create the sentence embeddings for the example product description
#query = "A handbag is a medium-to-large bag typically used by women to hold personal items. It is often fashionably designed. Versions of the term are 'purse', 'pocketbook', 'pouch', or 'clutch', terms which suggest rather smaller versions."
#query = 'A protection, safety, and private security agency. We specialize in the areas of close protection, property and home security and event security.'
query = 'A Home appliance is any consumer-electronic machine use to complete some household task, such as cooking or cleaning. Home appliances can be classified into: Major appliances (or white goods) and Small appliances'
#query = 'ovens for laboratory use'
#query = 'computer game software for use on mobile and cellular telephones'
query_vec = sbert_model.encode([query])[0]
#Calculate the similarity of the product description to the Nice classifications
ncl_sim = []
for ncl in sentence_embeddings:
ncl_sim.append(cosine(query_vec, ncl))
ncl_all['similarity'] = ncl_sim
#Display the top 20 matches
ncl_all.sort_values(by=['similarity'], ascending=False).head(20).style.set_properties(subset=['Desc'], **{'width-min': '50px'})
The dataset used was consolidated from various files found on the WIPO website and can be downloaded from here: https://github.com/bricesh/ncl_recommender/blob/9e9b1b8feef60ba71d84956a78f152c157508c2b/data/consolidated_nice_classifications.csv
The power of the SBERT is evident: “A Home appliance is any consumer-electronic machine use to complete some household task, such as cooking or cleaning…” is closely related to sentences such as “electric appliances for making yogurt, bread-making machines, coffee machines, ice-cream making machines” or “cleaning appliances utilizing steam” in its embedding space.
If you would like to explore the idea further, I wrote a more user-friendly app using Streamlit: https://bricesh-ncl-recommender-app-dyz6y3.streamlit.app/
Disclaimer
This work is an unfinished academic exercise not intended to inform real filing decisions. The trademark registration process can be complex and time-consuming. It is highly recommended to seek the advice of a qualified Trademark Attorney who would guide an applicant through the process and ensure that their application is completed properly.
