Associate-Developer-Apache-Spark最も有効な質問と解答で勉強

Which of the following code blocks returns a 2-column DataFrame that shows the distinct values in column productId and the number of rows with that productId in DataFrame transactionsDf?

• A. transactionsDf.groupBy("productId").agg(col("value").count())
• B. transactionsDf.groupBy("productId").count()
• C. transactionsDf.groupBy("productId").select(count("value"))
• D. transactionsDf.count("productId").distinct()
• E. transactionsDf.count("productId")

解説: (ShikenPASS メンバーにのみ表示されます)

The code block displayed below contains an error. The code block should return all rows of DataFrame transactionsDf, but including only columns storeId and predError. Find the error.
Code block:
spark.collect(transactionsDf.select("storeId", "predError"))

• A. The collect method is not a method of the SparkSession object.
• B. Instead of select, DataFrame transactionsDf needs to be filtered using the filter operator.
• C. Columns storeId and predError need to be represented as a Python list, so they need to be wrapped in brackets ([]).
• D. Instead of collect, collectAsRows needs to be called.
• E. The take method should be used instead of the collect method.

解説: (ShikenPASS メンバーにのみ表示されます)

Which of the following describes Spark's Adaptive Query Execution?

• A. Adaptive Query Execution is enabled in Spark by default.
• B. Adaptive Query Execution reoptimizes queries at execution points.
• C. Adaptive Query Execution features are dynamically switching join strategies and dynamically optimizing skew joins.
• D. Adaptive Query Execution features include dynamically coalescing shuffle partitions, dynamically injecting scan filters, and dynamically optimizing skew joins.
• E. Adaptive Query Execution applies to all kinds of queries.

解説: (ShikenPASS メンバーにのみ表示されます)

The code block displayed below contains an error. The code block should combine data from DataFrames itemsDf and transactionsDf, showing all rows of DataFrame itemsDf that have a matching value in column itemId with a value in column transactionsId of DataFrame transactionsDf. Find the error.
Code block:
itemsDf.join(itemsDf.itemId==transactionsDf.transactionId)

• A. The join statement is incomplete.
• B. The union method should be used instead of join.
• C. The join expression is malformed.
• D. The join method is inappropriate.
• E. The merge method should be used instead of join.

解説: (ShikenPASS メンバーにのみ表示されます)

Which of the following code blocks returns a new DataFrame with only columns predError and values of every second row of DataFrame transactionsDf?
Entire DataFrame transactionsDf:
1.+-------------+---------+-----+-------+---------+----+
2.|transactionId|predError|value|storeId|productId| f|
3.+-------------+---------+-----+-------+---------+----+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.| 4| null| null| 3| 2|null|
8.| 5| null| null| null| 2|null|
9.| 6| 3| 2| 25| 2|null|
10.+-------------+---------+-----+-------+---------+----+

• A. transactionsDf.filter(col(transactionId).isin([3,4,6]))
• B. 1.transactionsDf.createOrReplaceTempView("transactionsDf")
  2.spark.sql("FROM transactionsDf SELECT predError, value WHERE transactionId % 2 = 2")
• C. transactionsDf.filter(col("transactionId").isin([3,4,6])).select([predError, value])
• D. transactionsDf.select(col("transactionId").isin([3,4,6]), "predError", "value")
• E. transactionsDf.filter(col("transactionId") % 2 == 0).select("predError", "value") (Correct)
• F. transactionsDf.filter("transactionId" % 2 == 0).select("predError", "value")

解説: (ShikenPASS メンバーにのみ表示されます)