PROFESSIONAL-MACHINE-LEARNING-ENGINEER practice question 21 of 35

You are developing a mode! to detect fraudulent credit card transactions. You need to prioritize detection because missing even one fraudulent…

Full set: 283 questions
You are developing a mode! to detect fraudulent credit card transactions. You need to prioritize detection because missing even one fraudulent transaction could severely impact the credit card holder. You used AutoML to tram a model on users' profile information and credit card transaction dat a. After training the initial model, you notice that the model is failing to detect many fraudulent transactions. How should you adjust the training parameters in AutoML to improve model performance? Choose 2 answers
Answer options
A: Increase the score threshold.
B: Decrease the score threshold. Correct answer
C: Add more positive examples to the training set. Correct answer
D: Add more negative examples to the training set.
E: Reduce the maximum number of node hours for training.
Correct Answer:
B. Decrease the score threshold.; C. Add more positive examples to the training set.
Explanation

The primary goal is to minimize missed fraudulent transactions, which means maximizing recall (the proportion of actual positives that are correctly identified). The model is currently failing to detect many fraudulent transactions, indicating low recall.
1. Decreasing the score threshold (B) makes the model more sensitive. It will classify more transactions as fraudulent, thereby increasing the number of true positives (and recall) at the expense of increasing false positives (lower precision). This directly addresses the requirement to catch more fraudulent cases.
2. Adding more positive examples (C) helps the model learn the patterns of the underrepresented fraudulent class. Fraud detection datasets are typically highly imbalanced. Providing more examples of fraud helps mitigate this imbalance, leading to a model that is better at identifying the minority class.

Why Incorrect

A. Increase the score threshold: This would make the model less sensitive, requiring a higher confidence score to flag a transaction as fraudulent, thus increasing the number of missed fraudulent transactions (lower recall).
D. Add more negative examples to the training set: This would worsen the existing class imbalance, making it even harder for the model to learn the patterns of the rare fraudulent class.
E. Reduce the maximum number of node hours for training: This limits the time AutoML has to search for an optimal model, which is likely to result in a less accurate model and does not specifically address the low recall issue.

References

1. Google Cloud Documentation, "Evaluate your AutoML Tables model": In the section "Adjusting the classification threshold," the documentation states, "Lowering the threshold results in higher recall and lower precision. Raising the threshold results in lower recall and higher precision." This directly supports decreasing the threshold to improve fraud detection.

Source: Google Cloud. (n.d.). Evaluate your AutoML Tables model. Google Cloud Documentation. Retrieved from https://cloud.google.com/automl-tables/docs/evaluate#classification-threshold

2. Google Cloud Documentation, "Prepare your training data for AutoML Tables": The documentation on data distribution notes, "For classification models, if your training data is highly imbalanced, it can affect the model... Having too many of one class (the majority class) and not enough of another (the minority class) can cause the model to be biased toward the majority class." This supports the need to address imbalance, which can be done by adding more positive examples.

Source: Google Cloud. (n.d.). Prepare your training data for AutoML Tables. Google Cloud Documentation. Retrieved from https://cloud.google.com/automl-tables/docs/data-prep#distribution

3. Stanford University, CS229 Machine Learning Course Notes: In discussions on skewed classes and precision/recall, it is a standard principle that to increase recall, one can lower the classification threshold. It is also a common strategy to address class imbalance by collecting more data for the minority class.

Source: Ng, A. (n.d.). CS229 Machine Learning Course, Lecture Notes: Part VI - Learning Theory. Stanford University, p. 10-11. (Discusses precision-recall tradeoff and handling skewed classes).

Scroll to Top