file-type

MySQL代码实战:SQL第二次作业解析

ZIP文件

下载需积分: 8 | 718B | 更新于2024-12-31 | 29 浏览量 | 0 下载量 举报 收藏
download 立即下载
MySQL是一种广泛使用的开源关系数据库管理系统(RDBMS),其数据存储、操作、维护等方面的功能通过结构化查询语言(SQL)来实现。本次作业的文件内容涉及到MySQL数据库操作的相关知识点,下面将从文件名称出发,详细解读可能包含的数据库知识点。 ### 文件名:main.mysql 该文件名表明这是一个与MySQL相关的文件,很可能包含了数据库脚本或者SQL代码。根据标题与描述,文件可能包含以下几个方面的知识点: 1. **SQL基础语法**:包括数据定义语言(DDL),例如CREATE、ALTER、DROP语句来创建、修改和删除数据库对象;数据操作语言(DML),如INSERT、UPDATE、DELETE语句进行数据的增加、修改和删除;数据查询语言(DQL),例如SELECT语句来查询数据库中的数据。 2. **数据库设计**:在创建数据库时,需要考虑数据表的设计,这包括字段的设置、数据类型的定义、主键与外键的设置,以及必要的索引设计以提高查询效率。 3. **SQL高级特性**:比如子查询、联结(JOINs)、分组(GROUP BY)、排序(ORDER BY)、聚合函数(如COUNT、SUM、AVG、MIN、MAX)以及事务控制(如BEGIN、COMMIT、ROLLBACK)等。 4. **数据完整性与约束**:这可能包含对数据完整性的实现,如非空约束(NOT NULL)、唯一约束(UNIQUE)、主键约束(PRIMARY KEY)、外键约束(FOREIGN KEY)和检查约束(CHECK)等。 5. **性能优化**:可能包括了使用EXPLAIN语句进行SQL语句性能分析,以及对查询进行优化的相关技巧。 ### 文件名:README.txt 这个文件名暗示它是一个文本格式的说明文件,用来提供关于main.mysql文件的额外信息。该文件可能包含的知识点有: 1. **作业要求说明**:详细说明了本次作业的具体要求,如需要完成的SQL语句类型、设计的数据库结构、预期的数据量等。 2. **环境与配置说明**:可能涉及到本次作业运行所需的MySQL环境配置,包括但不限于MySQL的版本要求、所需安装的组件、连接信息等。 3. **提交方式说明**:可能描述了如何提交本次作业,包括文件提交的格式要求、截止时间、提交路径等。 4. **作业评分标准**:可能包含了教师或助教对作业进行评分的标准,帮助学生了解如何根据这些标准来完成作业。 5. **附加资源**:可能提供了一些与作业相关的附加资源,如教程链接、数据库设计的书籍推荐、其他辅助学习的资料等。 通过上述分析,可以发现本次作业涉及的MySQL知识点相当广泛,不仅包括基本的SQL语句编写和数据库操作,还可能包括数据库设计、高级查询优化以及性能调优等方面。对于学生而言,这是一个很好的实践机会,可以帮助他们巩固课堂上学到的理论知识,并将其应用于实际的数据库操作中。

相关推荐

filetype

一. mysqldump命令备份数据在MySQL中提供了命令行导出数据库数据以及文件的一种方便的工具mysqldump,我们可以通过命令行直接实现数据库内容的导出dump,首先我们简单了解一下mysqldump命令用法:#MySQLdump常用mysqldump -u root -p --databases 数据库1 数据库2 > xxx.sql二. mysqldump常用操作示例1.备份全部数据库的数据和结构mysqldump -uroot -p123456 -A > /data/mysqlDump/mydb.sql2.备份全部数据库的结构(加 -d 参数)mysqldump -uroot -p123456 -A -d > /data/mysqlDump/mydb.sql3.备份全部数据库的数据(加 -t 参数)mysqldump -uroot -p123456 -A -t > /data/mysqlDump/mydb.sql4.备份单个数据库的数据和结构(,数据库名mydb)mysqldump -uroot-p123456 mydb > /data/mysqlDump/mydb.sql5. 备份单个数据库的结构mysqldump -uroot -p123456 mydb -d > /data/mysqlDump/mydb.sql6. 备份单个数据库的数据mysqldump -uroot -p123456 mydb -t > /data/mysqlDump/mydb.sql7. 备份多个表的数据和结构(数据,结构的单独备份方法与上同)mysqldump -uroot -p123456 mydb t1 t2 > /data/mysqlDump/mydb.sql8. 一次备份多个数据库mysqldump -uroot -p123456 --databases db1 db2 > /data/mysqlDump/mydb.sql三. 还原 MySQL 备份内容有两种方式还原,第一种是在 MySQL 命令行中,第二种是使用 SHELL 行完成还原1.在系统命令行中,输入如下实现还原:mysql -uroot -p123456 < /data/mysqlDump/mydb.sql2.在登录进入mysql系统中,通过source指令找到对应系统中的文件进行还原:mysql> source /data/mysqlDump/mydb.sql在 Linux中,通常使用BASH脚本对需要执行的内容进行编写,加上定时执行命令crontab实现日志自动化生成。以下代码功能就是针对mysql进行备份,配合crontab,实现备份的内容为近一个月(31天)内的每天的mysql数据库记录。编写BASH维护固定数量备份文件在Linux中,使用vi或者vim编写脚本内容并命名为:mysql_dump_script.sh#!/bin/bash #保存备份个数,备份31天数据number=31#备份保存路径backup_dir=/root/mysqlbackup#日期dd=`date +%Y-%m-%d-%H-%M-%S`#备份工具tool=mysqldump#用户名username=root#密码password=TankB214#将要备份的数据库database_name=edoctor #如果文件夹不存在则创建if [ ! -d $backup_dir ];then         mkdir -p $backup_dir;fi #简单写法 mysqldump -u root -p123456 users > /root/mysqlbackup/users-$filename.sql$tool -u $username -p$password $database_name > $backup_dir/$database_name-$dd.sql #写创建备份日志echo "create $backup_dir/$database_name-$dd.dupm" >> $backup_dir/log.txt #找出需要删除的备份delfile=`ls -l -crt $backup_dir/*.sql | awk '{print $9 }' | head -1` #判断现在的备份数量是否大于$numbercount=`ls -l -crt $backup_dir/*.sql | awk '{print $9 }' | wc -l` if [ $count -gt $number ]then  #删除最早生成的备份,只保留number数量的备份  rm $delfile  #写删除文件日志  echo "delete $delfile" >> $backup_dir/log.txtfi如上代码主要含义如下:1.首先设置各项参数,例如number最多需要备份的数目,备份路径,用户名,密码等。2.执行mysqldump命令保存备份文件,并将操作打印至同目录下的log.txt中标记操作日志。3.定义需要删除的文件:通过ls命令获取第九列,即文件名列,再通过实现定义操作时间最晚的那个需要删除的文件。4.定义备份数量:通过ls命令加上wc -l统计以sql结尾的文件的行数。5.如果文件超出限制大小,就删除最早创建的sql文件使用crontab定期执行备份脚本在 Linux 中,周期执行的任务一般由cron这个守护进程来处理[ps -ef|grep cron]。cron读取一个或多个配置文件,这些配置文件中包含了命令行及其调用时间。cron的配置文件称为“crontab”,是“cron table”的简写。cron服务cron是一个 Liunx 下 的定时执行工具,可以在无需人工干预的情况下运行作业。service crond start    //启动服务service crond stop     //关闭服务service crond restart  //重启服务service crond reload   //重新载入配置service crond status   //查看服务状态 crontab语法crontab命令用于安装、删除或者列出用于驱动cron后台进程的表格。用户把需要执行的命令序列放到crontab文件中以获得执行。每个用户都可以有自己的crontab文件。/var/spool/cron下的crontab文件不可以直接创建或者直接修改。该crontab文件是通过crontab命令创建的。在crontab文件中如何输入需要执行的命令和时间。该文件中每行都包括六个域,其中前五个域是指定命令被执行的时间,最后一个域是要被执行的命令。每个域之间使用空格或者制表符分隔。格式如下:minute hour day-of-month month-of-year day-of-week commands 合法值 00-59 00-23 01-31 01-12 0-6 (0 is sunday)  除了数字还有几个个特殊的符号就是"*"、"/"和"-"、",",*代表所有的取值范围内的数字,"/"代表每的意思,"/5"表示每5个单位,"-"代表从某个数字到某个数字,","分开几个离散的数字。 -l 在标准输出上显示当前的crontab。 -r 删除当前的crontab文件。 -e 使用VISUAL或者EDITOR环境变量所指的编辑器编辑当前的crontab文件。当结束编辑离开时,编辑后的文件将自动安装。 创建cron脚本第一步:写cron脚本文件,命名为mysqlRollBack.cron。15,30,45,59 * * * * echo "xgmtest....." >> xgmtest.txt  表示,每隔15分钟,执行打印一次命令第二步:添加定时任务。执行命令 “crontab crontest.cron”。搞定第三步:"crontab -l" 查看定时任务是否成功或者检测/var/spool/cron下是否生成对应cron脚本注意:这操作是直接替换该用户下的crontab,而不是新增定期执行编写的定时任务脚本(记得先给shell脚本执行权限)0 2 * * * /root/mysql_backup_script.sh随后使用crontab命令定期指令编写的定时脚本crontab mysqlRollback.cron再通过命令检查定时任务是否已创建:附 crontab 的使用示例:1.每天早上6点0 6 * * * echo "Good morning." >> /tmp/test.txt //注意单纯echo,从屏幕上看不到任何输出,因为cron把任何输出都email到root的信箱了。2.每两个小时0 */2 * * * echo "Have a break now." >> /tmp/test.txt3.晚上11点到早上8点之间每两个小时和早上八点0 23-7/2,8 * * * echo "Have a good dream" >> /tmp/test.txt4.每个月的4号和每个礼拜的礼拜一到礼拜三的早上11点0 11 4 * 1-3 command line5.1月1日早上4点0 4 1 1 * command line SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root //如果出现错误,或者有数据输出,数据作为邮件发给这个帐号 HOME=/6.每小时执行/etc/cron.hourly内的脚本01 * * * * root run-parts /etc/cron.hourly7.每天执行/etc/cron.daily内的脚本02 4 * * * root run-parts /etc/cron.daily8.每星期执行/etc/cron.weekly内的脚本22 4 * * 0 root run-parts /etc/cron.weekly9.每月去执行/etc/cron.monthly内的脚本42 4 1 * * root run-parts /etc/cron.monthly注意: "run-parts" 这个参数了,如果去掉这个参数的话,后面就可以写要运行的某个脚本名,而不是文件夹名。10.每天的下午4点、5点、6点的5 min、15 min、25 min、35 min、45 min、55 min时执行命令。5,15,25,35,45,55 16,17,18 * * * command11.每周一,三,五的下午3:00系统进入维护状态,重新启动系统。00 15 * * 1,3,5 shutdown -r +512.每小时的10分,40分执行用户目录下的innd/bbslin这个指令:10,40 * * * * innd/bbslink13.每小时的1分执行用户目录下的bin/account这个指令:1 * * * * bin/account以下是我的测试每分钟的截图效果,其对应代码如下:* * * * * /root/mysql_backup_script.sh效果截图:其中的log.txt记录备份的操作详细日志:来源:cnblogs.com/letcafe/p/mysqlautodump.html

filetype

[root@node ~]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 8 Server version: 8.0.42 MySQL Community Server - GPL Copyright (c) 2000, 2025, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE DATABASE weblog_db; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE DATABASE weblog_db' at line 1 mysql> CREATE DATABASE weblog_db; ERROR 1007 (HY000): Can't create database 'weblog_db'; database exists mysql> USE weblog_db; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> DROP DATABASE IF EXISTS weblog_db; Query OK, 2 rows affected (0.02 sec) mysql> CREATE DATABASE weblog_db; Query OK, 1 row affected (0.01 sec) mysql> USE weblog_db; Database changed mysql> CREATE TABLE page_visits ( -> page VARCHAR(255) , -> visits BIGINT -> ); ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TABLE page_visits ( page VARCHAR(255) , visits BIGINT )' at line 1 mysql> CREATE TABLE page_visits ( -> page VARCHAR(255), -> visits BIGINT -> ); Query OK, 0 rows affected (0.02 sec) mysql> SHOW TABLES; +---------------------+ | Tables_in_weblog_db | +---------------------+ | page_visits | +---------------------+ 1 row in set (0.00 sec) mysql> ^C mysql> q -> quit -> exit -> ^C mysql> ^C mysql> ^C mysql> ^DBye [root@node ~]# hive SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Hive Session ID = 7bb79582-cc2b-49b6-abc7-020dcdc46542 Logging initialized using configuration in jar:file:/home/hive-3.1.3/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Hive Session ID = 15d9da52-e18e-40b2-a80f-e76eda81df4c hive> DESCRIBE FORMATTED page_visits; OK # col_name data_type comment page string visits bigint # Detailed Table Information Database: default OwnerType: USER Owner: root CreateTime: Tue Jul 08 01:43:42 CST 2025 LastAccessTime: UNKNOWN Retention: 0 Location: hdfs://node:9000/hive/warehouse/page_visits Table Type: MANAGED_TABLE Table Parameters: COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} bucketing_version 2 numFiles 1 numRows 4 rawDataSize 56 totalSize 60 transient_lastDdlTime 1751910222 # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.TextInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Storage Desc Params: serialization.format 1 Time taken: 0.785 seconds, Fetched: 32 row(s) hive> [root@node ~]# [root@node ~]# sqoop export \ > --connect jdbc:mysql://localhost/weblog_db \ > --username root \ > --password Aa@123456 \ > --table page_visits \ > --export-dir hdfs://node:9000/hive/warehouse/page_visits \ > --input-fields-terminated-by '\001' \ > --num-mappers 1 Warning: /home/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /home/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2025-07-08 15:28:12,550 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 2025-07-08 15:28:12,587 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-07-08 15:28:12,704 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-07-08 15:28:12,708 INFO tool.CodeGenTool: Beginning code generation Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2025-07-08 15:28:13,225 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `page_visits` AS t LIMIT 1 2025-07-08 15:28:13,266 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `page_visits` AS t LIMIT 1 2025-07-08 15:28:13,280 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/hadoop3.3 Note: /tmp/sqoop-root/compile/363869e21c2078b9742685122c43a3cc/page_visits.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 2025-07-08 15:28:16,377 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/363869e21c2078b9742685122c43a3cc/page_visits.jar 2025-07-08 15:28:16,391 INFO mapreduce.ExportJobBase: Beginning export of page_visits 2025-07-08 15:28:16,391 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-07-08 15:28:16,484 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-07-08 15:28:17,339 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2025-07-08 15:28:17,342 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-07-08 15:28:17,343 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2025-07-08 15:28:17,555 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at node/192.168.196.122:8032 2025-07-08 15:28:17,782 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/root/.staging/job_1751959003014_0001 2025-07-08 15:28:26,026 INFO input.FileInputFormat: Total input files to process : 1 2025-07-08 15:28:26,029 INFO input.FileInputFormat: Total input files to process : 1 2025-07-08 15:28:26,495 INFO mapreduce.JobSubmitter: number of splits:1 2025-07-08 15:28:26,528 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-07-08 15:28:26,619 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1751959003014_0001 2025-07-08 15:28:26,620 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2025-07-08 15:28:26,805 INFO conf.Configuration: resource-types.xml not found 2025-07-08 15:28:26,805 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 2025-07-08 15:28:27,226 INFO impl.YarnClientImpl: Submitted application application_1751959003014_0001 2025-07-08 15:28:27,264 INFO mapreduce.Job: The url to track the job: http://node:8088/proxy/application_1751959003014_0001/ 2025-07-08 15:28:27,264 INFO mapreduce.Job: Running job: job_1751959003014_0001 2025-07-08 15:28:34,334 INFO mapreduce.Job: Job job_1751959003014_0001 running in uber mode : false 2025-07-08 15:28:34,335 INFO mapreduce.Job: map 0% reduce 0% 2025-07-08 15:28:38,374 INFO mapreduce.Job: map 100% reduce 0% 2025-07-08 15:28:38,381 INFO mapreduce.Job: Job job_1751959003014_0001 failed with state FAILED due to: Task failed task_1751959003014_0001_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 2025-07-08 15:28:38,448 INFO mapreduce.Job: Counters: 8 Job Counters Failed map tasks=1 Launched map tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=2061 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=2061 Total vcore-milliseconds taken by all map tasks=2061 Total megabyte-milliseconds taken by all map tasks=2110464 2025-07-08 15:28:38,456 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2025-07-08 15:28:38,457 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 21.1033 seconds (0 bytes/sec) 2025-07-08 15:28:38,462 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2025-07-08 15:28:38,462 INFO mapreduce.ExportJobBase: Exported 0 records. 2025-07-08 15:28:38,462 ERROR mapreduce.ExportJobBase: Export job failed! 2025-07-08 15:28:38,463 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) [root@node ~]# sqoop export \ > --connect jdbc:mysql://localhost/weblog_db \ > --username root \ > --password Aa@123456 \ > --table page_visits \ > --export-dir hdfs://node:9000/hive/warehouse/page_visits \ > --input-fields-terminated-by ',' \ > --num-mappers 1 Warning: /home/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /home/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2025-07-08 15:30:31,174 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 2025-07-08 15:30:31,218 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-07-08 15:30:31,333 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-07-08 15:30:31,336 INFO tool.CodeGenTool: Beginning code generation Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2025-07-08 15:30:31,771 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `page_visits` AS t LIMIT 1 2025-07-08 15:30:31,814 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `page_visits` AS t LIMIT 1 2025-07-08 15:30:31,821 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/hadoop3.3 Note: /tmp/sqoop-root/compile/ab00e36d1f5084a0f7d522b4e9a975e5/page_visits.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 2025-07-08 15:30:33,116 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/ab00e36d1f5084a0f7d522b4e9a975e5/page_visits.jar 2025-07-08 15:30:33,129 INFO mapreduce.ExportJobBase: Beginning export of page_visits 2025-07-08 15:30:33,129 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-07-08 15:30:33,212 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-07-08 15:30:33,877 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2025-07-08 15:30:33,880 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-07-08 15:30:33,880 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2025-07-08 15:30:34,097 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at node/192.168.196.122:8032 2025-07-08 15:30:34,310 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/root/.staging/job_1751959003014_0002 2025-07-08 15:30:39,127 INFO input.FileInputFormat: Total input files to process : 1 2025-07-08 15:30:39,131 INFO input.FileInputFormat: Total input files to process : 1 2025-07-08 15:30:39,995 INFO mapreduce.JobSubmitter: number of splits:1 2025-07-08 15:30:40,022 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-07-08 15:30:40,532 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1751959003014_0002 2025-07-08 15:30:40,532 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2025-07-08 15:30:40,689 INFO conf.Configuration: resource-types.xml not found 2025-07-08 15:30:40,689 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 2025-07-08 15:30:40,746 INFO impl.YarnClientImpl: Submitted application application_1751959003014_0002 2025-07-08 15:30:40,783 INFO mapreduce.Job: The url to track the job: http://node:8088/proxy/application_1751959003014_0002/ 2025-07-08 15:30:40,784 INFO mapreduce.Job: Running job: job_1751959003014_0002 2025-07-08 15:30:46,847 INFO mapreduce.Job: Job job_1751959003014_0002 running in uber mode : false 2025-07-08 15:30:46,848 INFO mapreduce.Job: map 0% reduce 0% 2025-07-08 15:30:50,893 INFO mapreduce.Job: map 100% reduce 0% 2025-07-08 15:30:51,905 INFO mapreduce.Job: Job job_1751959003014_0002 failed with state FAILED due to: Task failed task_1751959003014_0002_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 2025-07-08 15:30:51,973 INFO mapreduce.Job: Counters: 8 Job Counters Failed map tasks=1 Launched map tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=2058 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=2058 Total vcore-milliseconds taken by all map tasks=2058 Total megabyte-milliseconds taken by all map tasks=2107392 2025-07-08 15:30:51,979 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2025-07-08 15:30:51,980 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 18.0828 seconds (0 bytes/sec) 2025-07-08 15:30:51,983 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2025-07-08 15:30:51,983 INFO mapreduce.ExportJobBase: Exported 0 records. 2025-07-08 15:30:51,983 ERROR mapreduce.ExportJobBase: Export job failed! 2025-07-08 15:30:51,984 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) [root@node ~]# 6.2 Sqoop导出数据 6.2.1从Hive将数据导出到MySQL 6.2.2sqoop导出格式 6.2.3导出page_visits表 6.2.4导出到ip_visits表 6.3验证导出数据 6.3.1登录MySQL 6.3.2执行查询

weixin_38637983
  • 粉丝: 8
上传资源 快速赚钱