PythonPandas两个表格内容模糊匹配的实现

PythonPandas两个表格内容模糊匹配的实现
⽬录
⼀、⽅法2
1. 导⼊库
2. 构建关键词
3. 构建句⼦
4. 建⽴统⼀索引
5. 表连接
6. 关键词匹配
⼆、⽅法2
1. 构建字典
2. 关键词匹配
3. 结果展⽰
4. 匹配结果展开
总结
⼀、⽅法2
此⽅法是两个表构建某⼀相同字段，然后全连接，在做匹配结果筛选，此⽅法针对数据量不⼤的时候，逻辑⽐较简单，但是内存消耗较⼤
1. 导⼊库
import pandas as pd
import numpy as np
import re
2. 构建关键词
#关键词数据
df_keyword = pd.DataFrame({
"keyid" : np.arange(5),
"keyword" : ["numpy", "pandas", "matplotlib", "sklearn", "tensorflow"]
})
df_keyword
3. 构建句⼦
df_sentence = pd.DataFrame({
"senid" : np.arange(10,17),
"sentence" : [
"怎样⽤pandas实现merge？",
"Python之Numpy详细教程",
"怎么使⽤Pandas批量拆分与合并Excel⽂件？",
"怎样使⽤pandas的map和apply函数？",
"深度学习之tensorflow简介",
"tensorflow和numpy的关系",
"基于sklearn的⼀些机器学习的代码"
]
})
df_sentence
4. 建⽴统⼀索引
df_keyword['match'] = 1
df_sentence['match'] = 1
5. 表连接
df_merge = pd.merge(df_keyword, df_sentence)
df_merge
6. 关键词匹配
def match_func(row):
return re.search(row["keyword"], row["sentence"], re.IGNORECASE) is not None df_merge[df_merge.apply(match_func, axis = 1)]
匹配结果如下
⼆、⽅法2
此⽅法对编程能⼒有要求，在⼤数据集上计算量较⽅法⼀⼩很多1. 构建字典
key_word_dict = {
row.keyword : row.keyid
for row in df_keyword.itertuples()
}
key_word_dict
{'numpy': 0, 'pandas': 1, 'matplotlib': 2, 'sklearn': 3, 'tensorflow': 4} 2. 关键词匹配
def merge_func(row):
#新增⼀列，表⽰可以匹配的keyid
row["keyids"] = [
keyid
for key_word, keyid in key_word_dict.items()
if re.search(key_word, row["sentence"], re.IGNORECASE)
]
return row
df_merge = df_sentence.apply(merge_func, axis = 1)
3. 结果展⽰
df_merge
4. 匹配结果展开
df_result = pd.merge(
left = df_merge.explode("keyids"),
right = df_keyword,
left_on = "keyids",
right_on = "keyid")
df_result
总结
到此这篇关于Python Pandas两个表格内容模糊匹配搜索以前的⽂章或继续浏览下⾯的相关⽂章希望⼤家以后多多⽀持！。