Large-scale Linear Models with TensorFlow#
TensorFlow下的大規(guī)模線性模型#
The tf.learn API provides (among other things) a rich set of tools for working with linear models in TensorFlow. This document provides an overview of those tools. It explains:
- what a linear model is.
- why you might want to use a linear model.
- how tf.learn makes it easy to build linear models in TensorFlow.
- how you can use tf.learn to combine linear models with deep learning to get the advantages of both.
在Tensorflow里，tf.learn API 提供一個豐富的用于線性模型的工具集。這篇文檔提供一個關于這些工具的概述，解釋了：
- 什么是線性模型。
- 為什么你可能會想使用線性模型醉冤。
- 如果在 Tensorflow 內(nèi)使 tf.learn 方便的建立線性模型。
- 如何使用 tf.learn 將線性模型和深度學習進行結(jié)合以獲得其優(yōu)勢黍氮。
Read this overview to decide whether the tf.learn linear model tools might be useful to you. Then do the Linear Models tutorial to give it a try. This overview uses code samples from the tutorial, but the tutorial walks through the code in greater detail.
閱讀這篇概述以決定 tf.learn 線性模型工具是否對你有用抹腿。然后嘗試一下 線性模型教程。這篇概述會使用該教程的示例代碼松却，并詳細介紹這些代碼暴浦。
To understand this overview it will help to have some familiarity with basic machine learning concepts, and also with tf.learn.
理解這篇概述，它將有助于你熟悉基本的機器學習的概念和 tf.learn.
Contents
- Large-scale Linear Models with TensorFlow
- What is a linear model?
- Why would you want to use a linear model?
- How does tf.learn help you build linear models?
目錄
What is a linear model?##
什么是線性模型砚哆？##
A linear model uses a single weighted sum of features to make a prediction. For example, if you have data on age, years of education, and weekly hours of work for a population, you can learn weights for each of those numbers so that their weighted sum estimates a person's salary. You can also use linear models for classification.
一個線性模型使用特征的單個加權和來進行預測独撇。例如，如果你有一些人口的年齡躁锁，受教育程度和每周工作時間的數(shù)據(jù)纷铣。你可以學習到每個參數(shù)，并使其的和來估計一個人的薪水战转。你也同樣可以使用線性模型來進行分類搜立。
Some linear models transform the weighted sum into a more convenient form. For example, logistic regression plugs the weighted sum into the logistic function to turn the output into a value between 0 and 1. But you still just have one weight for each input feature.
一些線性模型會將加權和轉(zhuǎn)換為一個更方便的形式。例如槐秧，logistic 回歸 注入加權和進 logistic函數(shù)中以將輸出轉(zhuǎn)換為0和1之間的值啄踊。但是你依然需要為每一個輸入的特征提供一個權重忧设。
Why would you want to use a linear model?##
為什么你想使用線性模型？##
Why would you want to use so simple a model when recent research has demonstrated the power of more complex neural networks with many layers?
為什么你要在最近的研究展示了多層復雜的神經(jīng)網(wǎng)絡的（強大）能力的情況下颠通，使用一個如此簡單的模型址晕？
Linear models:
- train quickly, compared to deep neural nets.
- can work well on very large feature sets.
- can be trained with algorithms that don't require a lot of fiddling with learning rates, etc.
- can be interpreted and debugged more easily than neural nets. You can examine the weights assigned to each feature to figure out what's having the biggest impact on a prediction.
- provide an excellent starting point for learning about machine learning.
- are widely used in industry.
線性模型:
- 相比于深度神經(jīng)網(wǎng)絡有著更快的訓練速度。
- 能夠在大量特征集下表現(xiàn)優(yōu)秀蒜哀。
- 能夠使用不需要大量的學習速率的算法進行訓練斩箫。
- 比起神經(jīng)網(wǎng)絡，容易進行解釋和調(diào)試撵儿。你可以為每個特征分配權重以獲得其對于預測有著多大的影響乘客。
- 為學習機器學習提供了一個極好的起步。
- 廣泛應用于工業(yè)淀歇。
How does tf.learn help you build linear models?##
tf.learn如何幫助你建立線性模型易核？###
You can build a linear model from scratch in TensorFlow without the help of a special API. But tf.learn provides some tools that make it easier to build effective large-scale linear models.
在Tensorflow下，你能夠在不借助一些特別的API的情況下浪默，從頭開始建立一個線性模型牡直。并且 tf.learn 提供一些工具使得能夠的建立一個大規(guī)模線性模型。
Feature columns and transformations###
特征列和轉(zhuǎn)換###
Much of the work of designing a linear model consists of transforming raw data into suitable input features. tf.learn uses the FeatureColumn abstraction to enable these transformations.
設計一個線性模型有許多工作纳决，包括轉(zhuǎn)換原始數(shù)據(jù)成一個適合的輸入特征碰逸。tf.learn 使用 FeatureColumn 來抽象使用這些變換。
A FeatureColumn represents a single feature in your data. A FeatureColumn may represent a quantity like 'height', or it may represent a category like 'eye_color' where the value is drawn from a set of discrete possibilities like {'blue', 'brown', 'green'}.
一個FeatureColumn 在你的數(shù)據(jù)中表示一個單個特征阔加。 一個 FeatyreColumn 可能會表示成 '高度' 的數(shù)量饵史，或者它可能表示成一個 '眼睛顏色' 的目錄并且其值可能會被繪制成一個像{'藍色', '棕色', '綠色'}的離散可能性值。
In the case of both continuous features like 'height' and categorical features like 'eye_color', a single value in the data might get transformed into a sequence of numbers before it is input into the model. The FeatureColumn abstraction lets you manipulate the feature as a single semantic unit in spite of this fact. You can specify transformations and select features to include without dealing with specific indices in the tensors you feed into the model.
在這兩種連續(xù)特征胜榔，像 '高度' 和 像 '眼睛顏色' 的 ‘分類特征’ 情況下胳喷，單個值在輸入這個模型前可能會被轉(zhuǎn)換成一個數(shù)字序列。抽象出的FeatureColumn 使得你能夠像操作語義一樣來操作這些特征夭织。你可以在不需要處理輸入模型的張量中的特定指數(shù)的情況下吭露，指定轉(zhuǎn)換方式并選擇要進行轉(zhuǎn)換的特征。
Sparse columns####
稀疏列####
Categorical features in linear models are typically translated into a sparse vector in which each possible value has a corresponding index or id. For example, if there are only three possible eye colors you can represent 'eye_color' as a length 3 vector: 'brown' would become [1, 0, 0], 'blue' would become [0, 1, 0] and 'green' would become [0, 0, 1]. These vectors are called "sparse" because they may be very long, with many zeros, when the set of possible values is very large (such as all English words).
分類特征在線性模型通常被轉(zhuǎn)化成一個稀疏向量尊惰，并且其對應值都有一個對應的序列或者id讲竿。例如，如果對于眼睛顏色只存在三種可能性的情況下弄屡，你可以表示‘眼睛顏色’為一個長度為3的向量：'棕色'應為[1, 0, 0], '藍色'應為[0, 1, 0]然后 '綠色' 為 [0, 0, 1]题禀。當可能值的集合非常大時（如所有英語單詞），這些向量被稱為“”稀疏向量”琢岩，因為他們可能很長但是存在很多0值投剥。
While you don't need to use sparse columns to use tf.learn linear models, one of the strengths of linear models is their ability to deal with large sparse vectors. Sparse features are a primary use case for the tf.learn linear model tools.
當你不需要使用稀疏來使用 tf.learn 的線性模型， 線性模型的其中一個優(yōu)勢是他有能力去處理大規(guī)模的稀疏向量担孔。稀疏特征是 tf.learn 線性模型工具的一個主要用例江锨。
Encoding sparse columns#####
編碼稀疏列#####
FeatureColumn handles the conversion of categorical values into vectors automatically, with code like this:
eye_color = tf.contrib.layers.sparse_column_with_keys( column_name="eye_color", keys=["blue", "brown", "green"])
稀疏列主要用于自動將分類值轉(zhuǎn)化為向量值吃警，其代碼像：
eye_color = tf.contrib.layers.sparse_column_with_keys( column_name="eye_color", keys=["blue", "brown", "green"])
where eye_color is the name of a column in your source data.
eye_color是你的源數(shù)據(jù)中一列的名稱。
You can also generate FeatureColumns for categorical features for which you don't know all possible values. For this case you would use sparse_column_with_hash_bucket(), which uses a hash function to assign indices to feature values.
education = tf.contrib.layers.sparse_column_with_hash_bucket("education", hash_bucket_size=1000)
當你不知道所有的可能值時啄育，你同樣能夠為分類特征生成FeatureColumns酌心。在這種情況下你應該使用sparse_column_with_hash_bucket()，其使用hash函數(shù)去為特征值分類序列挑豌。
education = tf.contrib.layers.sparse_column_with_hash_bucket("education", hash_bucket_size=1000)
Feature Crosses#####
交叉特征#####
Because linear models assign independent weights to separate features, they can't learn the relative importance of specific combinations of feature values. If you have a feature 'favorite_sport' and a feature 'home_city' and you're trying to predict whether a person likes to wear red, your linear model won't be able to learn that baseball fans from St. Louis especially like to wear red.
因為線性模型是給每個單獨的特征分配獨立的權重安券，所以他們無法學習特征在特定組合的重要性。如果你有一個特征 '最喜歡的運動' 和特征 '居住城市'氓英，然后并且你試圖想要推斷一個人在什么情況下會穿紅色的衣服侯勉。你的線性模型將沒有不會學習到來自St.Louis的棒球粉絲是很喜歡穿紅色系的衣服。
You can get around this limitation by creating a new feature 'favorite_sport_x_home_city'. The value of this feature for a given person is just the concatenation of the values of the two source features: 'baseball_x_stlouis', for example. This sort of combination feature is called a feature cross.
你可以通過創(chuàng)建一個新的叫‘最喜歡的運動x居住城市’ 特征來克服這個限制铝阐。這個特征的值對一個給定的人（的數(shù)據(jù)）來說只是關聯(lián)了兩個源特征的值址貌。例如，這種組合特征被稱為交叉特征徘键。
The crossed_column() method makes it easy to set up feature crosses:
sport = tf.contrib.layers.sparse_column_with_hash_bucket("sport", hash_bucket_size=1000) city = tf.contrib.layers.sparse_column_with_hash_bucket("city", hash_bucket_size=1000) sport_x_city = tf.contrib.layers.crossed_column([sport, city], hash_bucket_size=int(1e4))
crossed_column()函數(shù)能夠很容易就建立出交叉特征练对。
Continuous columns####
連續(xù)列####
You can specify a continuous feature like so:
age = tf.contrib.layers.real_valued_column("age")
你像這樣可以指定一個連續(xù)特征：
Although, as a single real number, a continuous feature can often be input directly into the model, tf.learn offers useful transformations for this sort of column as well.
盡管對于一個實數(shù)，一個連續(xù)的特征一般是能夠直接輸入到模型中的吹害。tf.learn也提供對連續(xù)列進行轉(zhuǎn)換螟凭。
Bucketization####
桶化####
Bucketization turns a continuous column into a categorical column. This transformation lets you use continuous features in feature crosses, or learn cases where specific value ranges have particular importance.
桶化轉(zhuǎn)換一個連續(xù)列為分類列。這種轉(zhuǎn)換會讓你在交叉特征中使用連續(xù)特征它呀，或者是學習對于特定的范圍值能夠造成的重大影響螺男。
Bucketization divides the range of possible values into subranges called buckets:
age_buckets = tf.contrib.layers.bucketized_column( age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
桶化會將可能值的范圍劃分為一系列小的子范圍，這樣的子范圍被稱之為桶：
The bucket into which a value falls becomes the categorical label for that value.
其桶內(nèi)新添加的值將成為該值的分類標簽
Input function####
輸入函數(shù)####
FeatureColumns provide a specification for the input data for your model, indicating how to represent and transform the data. But they do not provide the data itself. You provide the data through an input function.
FeatureColumns為你的模型輸入提供了一個規(guī)范格式钟些，表面如何去表示和轉(zhuǎn)換這些數(shù)據(jù)烟号。但是其本身不會提供這些數(shù)據(jù)绊谭。你需要通過一個輸入函數(shù)來提供這些數(shù)據(jù)政恍。
The input function must return a dictionary of tensors. Each key corresponds to the name of a FeatureColumn. Each key's value is a tensor containing the values of that feature for all data instances. See Building Input Functions with tf.contrib.learn for a more comprehensive look at input functions, and input_fn in the linear models tutorial code for an example implementation of an input function.
這個輸入函數(shù)必須返回一個張量的目錄。每一個鍵值對應一個FeatureColumn的名字达传。每一個鍵值對應的值是一個包含所有數(shù)據(jù)實例中某類特征值的張量篙耗。詳情可見使用tf.contrib.learn建立輸入函數(shù)以便更全面的了解輸入函數(shù)，并且在線性模型教程代碼中有引入了一個輸入函數(shù)的實例宪赶。
The input function is passed to the fit() and evaluate() calls that initiate training and testing, as described in the next section.
輸入函數(shù)通過調(diào)用fit()和evaluate()來發(fā)起訓練和測試宗弯，將會在下一節(jié)介紹他們。
Linear estimators###
線性估計器###
tf.learn's estimator classes provide a unified training and evaluation harness for regression and classification models. They take care of the details of the training and evaluation loops and allow the user to focus on model inputs and architecture.
tf.learn的估計器類為回歸和分類模型提供了一個統(tǒng)一的訓練和評估方法搂妻。他們主要處理訓練和評估循環(huán)的細節(jié)及其允許用戶能夠更專注于模型輸入與結(jié)構蒙保。
To build a linear estimator, you can use either the tf.contrib.learn.LinearClassifier estimator or the tf.contrib.learn.LinearRegressor estimator, for classification and regression respectively.
為了建立一個線性估計器，你可以使用分別用于分類和回歸的tf.contrib.learn.LinearClassifier估計器或tf.contrib.learn.LinearRegressor估計器欲主。
As with all tf.learn estimators, to run the estimator you just:
- Instantiate the estimator class. For the two linear estimator classes, you pass a list of FeatureColumns to the constructor.
- Call the estimator's fit() method to train it.
- Call the estimator's evaluate() method to see how it does.
For example:
e = tf.contrib.learn.LinearClassifier(feature_columns=[ native_country, education, occupation, workclass, marital_status, race, age_buckets, education_x_occupation, age_buckets_x_race_x_occupation], model_dir=YOUR_MODEL_DIRECTORY)
e.fit(input_fn=input_fn_train, steps=200)
# Evaluate for one step (one pass through the test data).
results = e.evaluate(input_fn=input_fn_test, steps=1)
# Print the stats for the evaluation.for key in sorted(results):
print "%s: %s" % (key, results[key])
所有的tf.learn估計器邓厕，只需要以下幾步就可以運行：
- 實例化估計器類逝嚎。對于這兩種估計器類，需要傳遞一個FeatureColumns列表到其構造器中详恼。
- 調(diào)用估計器的fit()方法去訓練它补君。
- 調(diào)用估計器的evaluate()方法去觀察其運行。
例如：
Wide and deep learning##
廣度和深度學習##
The tf.learn API also provides an estimator class that lets you jointly train a linear model and a deep neural network. This novel approach combines the ability of linear models to "memorize" key features with the generalization ability of neural nets. Use tf.contrib.learn.DNNLinearCombinedClassifier to create this sort of "wide and deep" model:
e = tf.contrib.learn.DNNLinearCombinedClassifier( model_dir=YOUR_MODEL_DIR, linear_feature_columns=wide_columns, dnn_feature_columns=deep_columns, dnn_hidden_units=[100, 50])
tf.learn API同樣提供一個估計器類使得讓你聯(lián)合訓練一個線性模型和一個深度神經(jīng)網(wǎng)絡昧互。這種新穎的方法結(jié)合了線性模型“記憶”關鍵特征與神經(jīng)網(wǎng)絡的泛化能力挽铁。使用tf.contrib.learn.DNNLinearCombinedClassifier去創(chuàng)建這種“廣度且深度”的模型：
For more information, see the Wide and Deep Learning tutorial.
欲了解更多信息，可以查看這個廣度和深度學習教程.
