0x01 NB

nnn
朴素:整个形式化的过程只做最原始、最简单的假设

优点

  • 数据较少情况下仍然有效
  • 可以处理多类别问题

缺点

  • 对于输入数据的处理方式比较敏感

适用数据类型

  • 标称型

0x02 贝叶斯决策理论

计算数据点属于每个类别的概率,并进行比较,选择具有最高概率的决策

条件概率

推导过程

0x03 构建文档分类器

两个假设

  • 特征之间相互独立(统计意义上的独立)
  • 每个特征同等重要

word2vec

def loadDataSet():
    '''
    测试数据
    :return: 
    '''
    postingList = [['my', 'dog', 'has', 'flea', 'problems', 'help', 'please'],
                 ['maybe', 'not', 'take', 'him', 'to', 'dog', 'park', 'stupid'],
                 ['my', 'dalmation', 'is', 'so', 'cute', 'I', 'love', 'him'],
                 ['stop', 'posting', 'stupid', 'worthless', 'garbage'],
                 ['mr', 'licks', 'ate', 'my', 'steak', 'how', 'to', 'stop', 'him'],
                 ['quit', 'buying', 'worthless', 'dog', 'food', 'stupid']]
    classVec = [0, 1, 0, 1, 0, 1]  #是否包含侮辱性词语,为1
    return postingList, classVec
                 
def createVocabList(dataSet):
    '''
    创建dataSet的不重复词列表
    :param dataSet: 
    :return: 
    '''
    vocabSet = set([])
    for document in dataSet:
        vocabSet = vocabSet | set(document)
    return list(vocabSet)

def setOfWords2Vec(vocabList, inputSet):
    '''
    :param vocabList: 不重复词列表
    :param inputSet: 某文档
    :return: 文档向量
    '''
    returnVec = [0]*len(vocabList) #创建一个长度和vocabList相等的全部为0的向量
    for word in inputSet:
        if word in vocabList:
            returnVec[vocabList.index(word)] = 1
        else:
            print("the word: %s is not in my Vocabulary!" % word)
    return returnVec #[0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]

训练算法

  • 从词向量计算概率
    for postdoc in postingList:
        trainmat.append(setOfWords2Vec(vocablist, postdoc))

通过setOfWords2Vec方法对文档进行处理,返回文档向量

def trainNB0(trainMatrix,trainCategory):
    numTrainDocs = len(trainMatrix) #6 文档矩阵的行数
    numWords = len(trainMatrix[0]) #32 矩阵的长度
    pAbusive = sum(trainCategory)/float(numTrainDocs) #3/6  文档属于侮辱类型的概率
    p0Num = ones(numWords) #ones函数可以创建任意维度和元素个数的数组,其元素值均为1
    p1Num = ones(numWords)
    p0Denom = 0.0
    p1Denom = 0.0
    for i in range(numTrainDocs):
        if trainCategory[i] == 1:
            p1Num += trainMatrix[i] #如果标签为侮辱性的,则两个列表相加
            p1Denom += sum(trainMatrix[i]) #侮辱性文档的词数相加
        else:
            p0Num += trainMatrix[i]
            p0Denom += sum(trainMatrix[i])
    #p1num:[2. 2. 1. 1. 1. 1. 2. 1. 1. 2. 2. 1. 1. 1. 4. 2. 3. 2. 1. 1. 1. 1. 2. 2.
 2. 2. 1. 1. 1. 2. 1. 3.]
    #p1Demon:19.0
    p1Vect = log(p1Num/p1Denom)          #将单个词的数目除以总词数得到条件概率
    p0Vect = log(p0Num/p0Denom)
    return p0Vect, p1Vect, pAbusive

概率向量:在给定文档类别条件下词汇表中单词的出现概率
p0Vect:正常文档的概率向量
p1Vect:侮辱性文档概率向量
pAbusive:侮辱文档的概率

  • 概率值为0问题

利用贝叶斯分类器对文档进行分类时,要计算多个概率的乘积以获得文档属于某个类别的概率,即计算p(w0|1)p(w1|1)p(w2|1)。如果其中一个概率值为0,那么最后的乘积也为0。为降低 这种影响,可以将所有词的出现数初始化为1,并将分母初始化为2

p0Denom = 2.0
p1Denom = 2.0
  • 下溢出问题

相乘许多很小的数,最后四舍五入后会得到0

p(w0|ci)*p(w1|ci)*...*p(w0|ci) 取对数,得到ln(p(w0|ci))+ln(p(w1|ci))+...+ln(p(w0|ci))

p1Vect = log(p1Num/p1Denom)         
p0Vect = log(p0Num/p0Denom)

测试算法


的含义为给定w向量的基础上来自类别ci的概率是多少

p(ci)的概率为pAbusive
接下来需要计算p(w|ci),假设所有词都互相独立,即
p(w0,w1,w2..wN|ci)=p(w0|ci)p(w1|ci)p(w2|ci)...p(wN|ci)

因为P(w)P(ci)两者是一样的,可以忽略

因为log(p(w|c)p(c)) = log(p(w|c)) + log(p(c)),所以在classifyNB方法中求和

def classifyNB(vec2Classify, p0Vec, p1Vec, pClass1):
    '''
    元素相乘
    :param vec2Classify:要分类的向量
    :param p0Vec:正常文档概率向量
    :param p1Vec:侮辱文档概率向量
    :param pClass1:侮辱文档的概率
    :return:1 or 0
    '''
    p1 = sum(vec2Classify * p1Vec) + log(pClass1)
    p0 = sum(vec2Classify * p0Vec) + log(1.0 - pClass1)
    if p1 > p0:
        return 1
    else: 
        return 0

便利函数

def testingNB():
    listOPosts,listClasses = loadDataSet()
    myVocabList = createVocabList(listOPosts)
    trainMat=[]
    for postinDoc in listOPosts:
        trainMat.append(setOfWords2Vec(myVocabList, postinDoc))
    p0V,p1V,pAb = trainNB0(array(trainMat), array(listClasses))
    testEntry = ['love', 'my', 'dalmation']
    thisDoc = array(setOfWords2Vec(myVocabList, testEntry))
    print(testEntry, 'classified as: ', classifyNB(thisDoc, p0V, p1V, pAb))
    testEntry = ['stupid', 'garbage']
    thisDoc = array(setOfWords2Vec(myVocabList, testEntry))
    print(testEntry, 'classified as: ', classifyNB(thisDoc, p0V, p1V, pAb))

  • 词袋模型

在词袋中,每个单词可以出现 多次,而在词集中,每个词只能出现一次

每当遇到一个单词时,词向量中的对应值会+1

def bagOfWords2VecMN(vocabList, inputSet):
    returnVec = [0]*len(vocabList)
    for word in inputSet:
        if word in vocabList:
            returnVec[vocabList.index(word)] += 1
    return returnVec

0x04 Action 1

垃圾邮件判断

def textParse(bigString):
    '''
    简单分词处理
    :param bigString: 
    :return: 
    '''
    import re
    listOfTokens = re.split('\W*', bigString)
    return [tok.lower() for tok in listOfTokens if len(tok) > 2] #取长度大于3,转化为小写
    
def spamTest():
    '''
    数据输入
    处理
    分割
    训练
    测试
    :return: 
    '''
    docList=[]
    classList = []
    fullText = []
    for i in range(1, 26):
        wordList = textParse(open('email/spam/%d.txt' % i, 'rb').read().decode('GBK', 'ignore'))
        docList.append(wordList)
        fullText.extend(wordList)
        classList.append(1)
        wordList = textParse(open('email/ham/%d.txt' % i, 'rb').read().decode('GBK', 'ignore'))
        docList.append(wordList)
        fullText.extend(wordList)
        classList.append(0)
    vocabList = createVocabList(docList) #创建不重复词表
    trainingSet = list(range(50)) #[0, 1, 2, 3, 4, 5, 6, 7, 8...44, 45, 46, 47, 48, 49]
    testSet=[]
    for i in range(10): #随机选择10条数据作为测试集
        randIndex = int(random.uniform(0, len(trainingSet)))
        testSet.append(trainingSet[randIndex])
        del(trainingSet[randIndex])  
    trainMat = []
    trainClasses = []
    for docIndex in trainingSet: # 训练集
        trainMat.append(bagOfWords2VecMN(vocabList, docList[docIndex])) #词袋模型,构建词向量
        trainClasses.append(classList[docIndex])
    p0V, p1V, pSpam = trainNB0(array(trainMat), array(trainClasses))
    errorCount = 0
    for docIndex in testSet: # 测试集
        wordVector = bagOfWords2VecMN(vocabList, docList[docIndex])
        if classifyNB(array(wordVector), p0V, p1V, pSpam) != classList[docIndex]:
            errorCount += 1
            print("classification error", docList[docIndex])
    print('the error rate is: ', float(errorCount)/len(testSet))

参考

[1] https://www.manning.com/books/machine-learning-in-action