最も有効な Associate-Developer-Apache-Spark 試験参考書

Which of the following code blocks creates a new DataFrame with two columns season and wind_speed_ms where column season is of data type string and column wind_speed_ms is of data type double?

• A. CharType()), T.StructField("season", T.DoubleType())]))
• B. spark.DataFrame({"season": ["winter","summer"], "wind_speed_ms": [4.5, 7.5]})
• C. spark.createDataFrame([("summer", 4.5), ("winter", 7.5)], ["season", "wind_speed_ms"])
• D. spark.createDataFrame({"season": ["winter","summer"], "wind_speed_ms": [4.5, 7.5]})
• E. spark.newDataFrame([("summer", 4.5), ("winter", 7.5)], ["season", "wind_speed_ms"])
• F. 1. from pyspark.sql import types as T
  2. spark.createDataFrame((("summer", 4.5), ("winter", 7.5)), T.StructType([T.StructField("season",

Which of the following statements about storage levels is incorrect?

• A. DISK_ONLY will not use the worker node's memory.
• B. Caching can be undone using the DataFrame.unpersist() operator.
• C. In client mode, DataFrames cached with the MEMORY_ONLY_2 level will not be stored in the edge node's memory.
• D. The cache operator on DataFrames is evaluated like a transformation.
• E. MEMORY_AND_DISK replicates cached DataFrames both on memory and disk.

Which of the following code blocks returns a DataFrame that has all columns of DataFrame transactionsDf and an additional column predErrorSquared which is the squared value of column predError in DataFrame transactionsDf?

• A. transactionsDf.withColumnRenamed("predErrorSquared", pow(predError, 2))
• B. transactionsDf.withColumn("predErrorSquared", "predError"**2)
• C. transactionsDf.withColumn("predError", pow(col("predErrorSquared"), 2))
• D. transactionsDf.withColumn("predErrorSquared", pow(col("predError"), lit(2)))
• E. transactionsDf.withColumn("predErrorSquared", pow(predError, lit(2)))

The code block shown below should return a copy of DataFrame transactionsDf without columns value and productId and with an additional column associateId that has the value 5. Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__(__2__, __3__).__4__(__5__, 'value')

• A. 1. withColumn
  2. 'associateId'
  3. lit(5)
  4. drop
  5. 'productId'
• B. 1. withColumnRenamed
  2. 'associateId'
  3. 5
  4. drop
  5. 'productId'
• C. 1. withNewColumn
  2. associateId
  3. lit(5)
  4. drop
  5. productId
• D. 1. withColumn
  2. col(associateId)
  3. lit(5)
  4. drop
  5. col(productId)
• E. 1. withColumn
  2. 'associateId'
  3. 5
  4. remove
  5. 'productId'

Which of the following code blocks returns all unique values across all values in columns value and productId in DataFrame transactionsDf in a one-column DataFrame?

• A. transactionsDf.select('value').union(transactionsDf.select('productId')).distinct()
• B. transactionsDf.select('value', 'productId').distinct()
• C. transactionsDf.agg({'value': 'collect_set', 'productId': 'collect_set'})
• D. tranactionsDf.select('value').join(transactionsDf.select('productId'), col('value')==col('productId'),
  'outer')
• E. transactionsDf.select(col('value'), col('productId')).agg({'*': 'count'})