跟人整理hadoop学习心得

hadoop 是实现了google的MapReduce的一个java类，MapReduce是一个中简化分布式编程模式

MapReduce框架提供map 和reduce两个关键编程接口
map 是用来打散数据，reduce用来将数据进行汇总

hadoop并不仅仅是一个用于存储的分布式文件系统，而是设计用来在由通用技术设备组成的大型集群上集群上执行分布式应用的框架。

作为hadoop程序员
1.定义Mapper，处理输入的Key-Value对，输出中间结果
2.定义reducer,可选，对中间结果进行规约，输出最终结果
3.定义inputFormat和OutputFormat,可选，InputFormat 将每行输入文件的内容转换为Java类供Mapper函数的使用，不定义时默认为String
4.定义main函数，在里面定义一个job并运行它，然后事情就交给系统，hadoo入门首先要了解一下基本概念

1.Hadoop的HDFS实现了google的GFS文件系统，NameNode作为文件系统的负责调度运行在master，DataNode运行在每个机器上，同时Hadoop实现了goole的Mapreduce,JobTracker作为MapReduce的总调度运行在master,TaskTrac
ker则运行在每个机器上执行Task

2.main()函数，创建JobConf,定义Mapper,Reducer,Input/OutputFormat和输入输出文件目录，最后把Job提交给JobTracker,等待job结束

3.JobTracker,创建一个InputFormat的实例，调用getSplits()方法，把输入目录的文件拆分成FileSplist作为Mappertask的输入，生产Mappertask加入Queue

4.TaskTracker向JobTracker索求一个Map/Reduce


BigTable。一个大型的分布式数据，用来存储结构化数据


单词过来频率




java虚拟机序列号： JHXUR-G0M88-GA44V-4MRN6

/item.htm?id=9782434796

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
下面，增加一个键，名字就用你的程序的名字，然后值是运行的命令行。



/WebServices/WeatherWebService.asmx?wsdl

/item.htm?id=10323285996&ad_id=&am_id=&cm_id=&pm_id=






正在 Ping 123.125.156.90 具有 32 字节的数据:请求超时。来自 123.125.156.90 的回复: 字节=32 时间=207ms TTL=47来自 123.125.156.90 的回复: 字节=32 时间=208ms TTL=47请求超时。123.125.156.90 的 Ping 统计信息: 数据包: 已发送 = 4，已接收 = 2，丢失 = 2 (50% 丢失)，往返行程的估计时间(以毫秒为单位): 最短 = 207ms，最长 = 208ms，平均 = 207ms
1成功生成（总时间：18 秒）

正在 Ping 123.125.156.90 具有 32 字节的数据:来自 192.168.1.10 的回复: 无法访问目标主机。来自 192.168.1.10 的回复: 无法访问目标主机。来自 192.168.1.10 的回复: 无法访问目标主机。来自 192.168.1.10 的回复: 无法访问目标主机。123.125.156.90 的 Ping 统计信息: 数据包: 已发送 = 4，已接收 = 4，丢失 = 0 (0% 丢失)，
SetOutPath "$INSTDIR\jre\bin"

File "jre\bin\awt.dll"

File "jre\bin\axbridge.dll"
SetOutPath "$INSTDIR\jre\bin\client"
File "jre\bin\client\jvm.dll"
SetOutPath "$INSTDIR\jre\bin"
File "jre\bin\dcpr.dll"
File "jre\bin\deploy.dll"
File "jre\bin\dt_shmem.dll"
File "jre\bin\dt_socket.dll"
File "jre\bin\fontmanager.dll"
File "jre\bin\hpi.dll"
File "jre\bin\hprof.dll"
File "jre\bin\instrument.dll"
File "jre\bin\ioser12.dll"
File "jre\bin\java.dll"
File "jre\bin\java.exe"
File "jre\bin\javaw.exe"
File "jre\bin\jawt.dll"
File "jre\bin\msvcr71.dll"
File "jre\bin\net.dll"
File "jre\bin\nio.dll"
File "jre\bin\npt.dll"
File "jre\bin\unpack.dll"
File "jre\bin\verify.dll"
File "jre\bin\zip.dll"
SetOutPath "$INSTDIR\jre\lib"
File "jre\lib\charsets.jar.pack"
SetOutPath "$INSTDIR\jre\lib\ext"
File "jre\lib\ext\localedata.jar"
File "jre\lib\ext\sunjce_provider.jar.pack"
SetOutPath "$INSTDIR\jre\lib"
File "jre\lib\fontconfig.bfc"
SetOutPath "$INSTDIR\jre\lib\i386"
File "jre\lib\i386\jvm.cfg"
SetOutPath "$INSTDIR\jre\lib"
File "jre\lib\jce.jar.pack"
File "jre\lib\jsse.jar.pack"
File "jre\lib\resources.jar"
File "jre\lib\rt.jar.pack"
SetOutPath "$INSTDIR\jre\lib\security"
File "jre\lib\security\cacerts"
File "jre\lib\security\java.policy"
File "jre\lib\security\java.security"
File "jre\lib\security\javaws.policy"
File "jre\lib\security\local_policy.jar"
File "jre\lib\security\US_export_policy.jar"
SetOutPath "$INSTDIR\jre\lib"
File "jre\lib\tzmappings"
SetOutPath "$INSTDIR\jre\lib\zi\Asia"
File "jre\lib\zi\Asia\Aden"
File "jre\lib\zi\Asia\Almaty"
File "jre\lib\zi\Asia\Amman"
File "jre\lib\zi\Asia\Anadyr"
File "jre\lib\zi\Asia\Aqtau"
File "jre\lib\zi\Asia\Aqtobe"
File "jre\lib\zi\Asia\Ashgabat"
File "jre\lib\zi\Asia\Baghdad"
File "jre\lib\zi\Asia\Bahrain"
File "jre\lib\zi\Asia\Baku"
File "jre\lib\zi\Asia\Bangkok"
File "jre\lib\zi\Asia\Beirut"
File "jre\lib\zi\Asia\Bishkek"
File "jre\lib\zi\Asia\Brunei"
File "jre\lib\zi\Asia\Calcutta"
File "jre\lib\zi\Asia\Choibalsan"
File "jre\lib\zi\Asia\Chongqing"
File "jre\lib\zi\Asia\Colombo"
File "jre\lib\zi\Asia\Damascus"
File "jre\lib\zi\Asia\Dhaka"
File "jre\lib\zi\Asia\Dili"
File "jre\lib\zi\Asia\Dubai"
File "jre\lib\zi\Asia\Dushanbe"
File "jre\lib\zi\Asia\Gaza"
File "jre\lib\zi\Asia\Harbin"
File "jre\lib\zi\Asia\Hong_Kong"
File "jre\lib\zi\Asia\Hovd"
File "jre\lib\zi\Asia\Irkutsk"
File "jre\lib\zi\Asia\Jakarta"
File "jre\lib\zi\Asia\Jayapura"
File "jre\lib\zi\Asia\Jerusalem"
File "jre\lib\zi\Asia\Kabul"
File "jre\lib\zi\Asia\Kamchatka"
File "jre\lib\zi\Asia\Karachi"
File "jre\lib\zi\Asia\Kashgar"
File "jre\lib\zi\Asia\Katmandu"
File "jre\lib\zi\Asia\Krasnoyarsk"
File "jre\lib\zi\Asia\Kuala_Lumpur"
File "jre\lib\zi\Asia\Kuching"
File "jre\lib\zi\Asia\Kuwait"
File "jre\lib\zi\Asia\Macau"
File "jre\lib\zi\Asia\Magadan"

File "jre\lib\zi\Asia\Makassar"
File "jre\lib\zi\Asia\Manila"
File "jre\lib\zi\Asia\Muscat"
File "jre\lib\zi\Asia\Nicosia"
File "jre\lib\zi\Asia\Novosibirsk"
File "jre\lib\zi\Asia\Omsk"
File "jre\lib\zi\Asia\Oral"
File "jre\lib\zi\Asia\Phnom_Penh"
File "jre\lib\zi\Asia\Pontianak"
File "jre\lib\zi\Asia\Pyongyang"
File "jre\lib\zi\Asia\Qatar"
File "jre\lib\zi\Asia\Qyzylorda"
File "jre\lib\zi\Asia\Rangoon"
File "jre\lib\zi\Asia\Riyadh"
File "jre\lib\zi\Asia\Riyadh87"
File "jre\lib\zi\Asia\Riyadh88"
File "jre\lib\zi\Asia\Riyadh89"
File "jre\lib\zi\Asia\Saigon"
File "jre\lib\zi\Asia\Sakhalin"
File "jre\lib\zi\Asia\Samarkand"
File "jre\lib\zi\Asia\Seoul"
File "jre\lib\zi\Asia\Shanghai"
File "jre\lib\zi\Asia\Singapore"
File "jre\lib\zi\Asia\Taipei"
File "jre\lib\zi\Asia\Tashkent"
File "jre\lib\zi\Asia\Tbilisi"
File "jre\lib\zi\Asia\Tehran"
File "jre\lib\zi\Asia\Thimphu"
File "jre\lib\zi\Asia\Tokyo"
File "jre\lib\zi\Asia\Ulaanbaatar"
File "jre\lib\zi\Asia\Urumqi"
File "jre\lib\zi\Asia\Vientiane"
File "jre\lib\zi\Asia\Vladivostok"
File "jre\lib\zi\Asia\Yakutsk"
File "jre\lib\zi\Asia\Yekaterinburg"
File "jre\lib\zi\Asia\Yerevan"
SetOutPath "$INSTDIR\jre\lib\zi\Etc"
File "jre\lib\zi\Etc\GMT"
File "jre\lib\zi\Etc\GMT+1"
File "jre\lib\zi\Etc\GMT+10"
File "jre\lib\zi\Etc\GMT+11"
File "jre\lib\zi\Etc\GMT+12"
File "jre\lib\zi\Etc\GMT+2"
File "jre\lib\zi\Etc\GMT+3"
File "jre\lib\zi\Etc\GMT+4"
File "jre\lib\zi\Etc\GMT+5"
File "jre\lib\zi\Etc\GMT+6"
File "jre\lib\zi\Etc\GMT+7"
File "jre\lib\zi\Etc\GMT+8"
File "jre\lib\zi\Etc\GMT+9"
File "jre\lib\zi\Etc\GMT-1"
File "jre\lib\zi\Etc\GMT-10"
File "jre\lib\zi\Etc\GMT-11"
File "jre\lib\zi\Etc\GMT-12"
File "jre\lib\zi\Etc\GMT-13"
File "jre\lib\zi\Etc\GMT-14"
File "jre\lib\zi\Etc\GMT-2"
File "jre\lib\zi\Etc\GMT-3"
File "jre\lib\zi\Etc\GMT-4"
File "jre\lib\zi\Etc\GMT-5"
File "jre\lib\zi\Etc\GMT-6"
File "jre\lib\zi\Etc\GMT-7"
File "jre\lib\zi\Etc\GMT-8"
File "jre\lib\zi\Etc\GMT-9"
File "jre\lib\zi\Etc\UCT"
File "jre\lib\zi\Etc\UTC"
SetOutPath "$INSTDIR\jre\lib\zi"
File "jre\lib\zi\GMT"
SetOutPath "$INSTDIR\jre\lib\zi\Pacific"
File "jre\lib\zi\Pacific\Apia"
File "jre\lib\zi\Pacific\Auckland"
File "jre\lib\zi\Pacific\Chatham"
File "jre\lib\zi\Pacific\Easter"
File "jre\lib\zi\Pacific\Efate"
File "jre\lib\zi\Pacific\Enderbury"
File "jre\lib\zi\Pacific\Fakaofo"
File "jre\lib\zi\Pacific\Fiji"
File "jre\lib\zi\Pacific\Funafuti"
File "jre\lib\zi\Pacific\Galapagos"
File "jre\lib\zi\Pacific\Gambier"
File "jre\lib\zi\Pacific\Guadalcanal"
File "jre\lib\zi\Pacific\Guam"
File "jre\lib\zi\Pacific\Honolulu"
File "jre\lib\zi\Pacific\Johnston"
File "jre\lib\zi\Pacific\Kiritimati"
File "jre\lib\zi\Pacific\Kosrae"
File "jre\lib\zi\Pacific\Kwajalein"
File "jre\lib\zi\Pacific\Majuro"

File "jre\lib\zi\Pacific\Marquesas"
File "jre\lib\zi\Pacific\Midway"
File "jre\lib\zi\Pacific\Nauru"
File "jre\lib\zi\Pacific\Niue"
File "jre\lib\zi\Pacific\Norfolk"
File "jre\lib\zi\Pacific\Noumea"
File "jre\lib\zi\Pacific\Pago_Pago"
File "jre\lib\zi\Pacific\Palau"
File "jre\lib\zi\Pacific\Pitcairn"
File "jre\lib\zi\Pacific\Ponape"
File "jre\lib\zi\Pacific\Port_Moresby"
File "jre\lib\zi\Pacific\Rarotonga"
File "jre\lib\zi\Pacific\Saipan"
File "jre\lib\zi\Pacific\Tahiti"
File "jre\lib\zi\Pacific\Tarawa"
File "jre\lib\zi\Pacific\Tongatapu"
File "jre\lib\zi\Pacific\Truk"
File "jre\lib\zi\Pacific\Wake"
File "jre\lib\zi\Pacific\Wallis"
SetOutPath "$INSTDIR\jre\lib\zi"
File "jre\lib\zi\ZoneInfoMappings"