1 Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs
——CMU CVPR2018
1.1 任务描述
total label: n (okapi/ zebra/ deer)
seen label: m (zebra/ deer) + training pictures
unseen label: n-m (okapi)
knowledge graph：seen + unseen
1.2 模型框架
Input：Word Embedding x_i （a semantic description of each specific class）
Output：visual classifier (logistic regression model) \hat{w}_i (GT w_i(i=1,...,m) by training data)
visual feature (by pre-trained ConvNet) * classifier = classification score
1.3 实验结果
效果翻两倍(๑•̀ㅂ•́)و✧
Code https://github.com/JudyYe/zero-shot-gcn
2 Rethinking Knowledge Graph Propagation for Zero-Shot Learning
2019.3.27 论文1的进阶版
2.1 主要思路
论文1中的6层GCN会造成over-smoothing的问题，所以本文提出了Dense Graph Propagation (DGP) 模块，直接连接远距离的结点，只需1层GCN。
2.2 模型框架
Dense Graph Propagation
2 connect patterns: descendant propagation & ancestor propagation
Training:
1. train the DGP to predict the last layer weights of a pre-trained CNN
2. train the CNN by optimizing the cross-entropy classification loss on the seen classes
Test:
CNN Feature Extraction * \wave{W}
2.3 实验结果
比文章1的结果更好(๑•̀ㅂ•́)و✧
3 The More You Know: Using Knowledge Graphs for Image Classification
——CMU CVPR2017
As most knowledge graphs are large for reasoning, [60] selects some related entities to build a sub-graph based on the result of object detection and applies GGNN to the extracted graph for prediction.
Graph model: GGNN -> GSNN
4 Multi-Label Zero-Shot Learning with Structured Knowledge Graphs
——CMU CVPR2018
4.1 新的知识图谱
Source: WordNet (easily accessible and contains rich semantic relationships between different concepts)
边种类：
1 super-subordinate: directly extracted from WordNet
2 positive correlation & negative correlation: label similarities are calculated by WUP similarity [45], followed by thresholding the soft similarities into positive and negative correlations.
4.2 模型结构
1. 正常的multi-label分类作为初始化 x features -> Fi = node initial belief states hv^0
y \in {0,1}^{|S|}
2. label之间开始传递 Graph model：GGNN->GSNN
边W word embedding: F_R^k (w_u,w_v)->a_vu propagation weight (k=3)
3. 输出 hv^(T) -> Fo ->\hat{y}\in {0,1}
propagation matrix A
4.3 实验结果
5 总结
论文中的知识图谱的结构都是事先规定好的。
5.1 知识图谱数据库
语言知识图谱：WordNet
事实性知识图谱：OpenCyc , Freebase , DBpedia , YAGO2
领域知识图谱：人物之间的亲属关系Kinships，医学领域UMLS，Cora
机器自动构建的知识图谱：Knowledge Vault，NELL
统一医学语言系统(Unified Medical Language System，UMLS)是美国国立医学图书馆持续开发了20多年的巨型医学术语系统，涵盖了临床、基础、药学、生物学、医学管理等医学及与医学相关学科，收录了约200万个医学概念，医学词汇更是空前，达到了500多万个。
医疗图像具有特定性，比如WSI，就是癌细胞和非癌细胞两种，不像自然图片会有那么多种类。
准确性还比较低，医疗不太能允许有那么低的准确率
土拨鼠自己做笔记用的，不接受其他小鼠的批评～～
土拨鼠自己做笔记用的，不接受其他小鼠的批评～～
土拨鼠自己做笔记用的，不接受其他小鼠的批评～～
