Configure IKEv1 xAuth IPsec VPN Server with PSK or RSA in pfSense

The pfSense operating system allows us to configure different types of VPN, among which is IPsec IKEv1 xAuth, a type of VPN that will allow us to connect with our smartphones and tablets with Android and iOS, and also with our computers or Mac, as long as we install the corresponding software compatible with this type of protocol. Today in this article we are going to show you how to configure an IPsec IKEv1 xAuth VPN server so that you can connect remotely to your local network safely.

Configure IKEv1 xAuth IPsec VPN Server

What is an IPsec IKEv1 xAuth VPN server for?

The IPsec protocol is one of the most widely used and well-known VPN protocols, it is used both at the domestic level to connect us to our home easily and quickly (routers such as AVM FRITZ! Incorporate this type of secure VPN), as well as to connect us at the business level to the office or small and medium business. This VPN makes use of the IKEv1 (Internet Key Exchange) protocol to negotiate the encryption between the different clients and the server, in addition, it makes use of authentication with username and password (xAuth) that will allow us to authenticate the different clients when they connect to the server VPN. Within the IPsec IKEv1 xAuth protocol, we have two authentication methods:

  • Mutual PSK + xAuth: a password is established that each and every VPN client that wants to connect will have, this key is both on the server and on all clients. Subsequently, a username and password are requested to authenticate individually to the VPN.
  • Mutual RSA + xAuth: a CA must be created with server certificates and also certificates for the VPN clients, once the authentication with these certificates has been established, a username and password will also be required to authenticate in the VPN.

A very important detail is that we will not be able to use the IKEv2 protocol since it is not compatible with this type of VPN, we must use yes or yes IKEv1 so that VPN clients can connect correctly.

In this tutorial we will see how to configure the IPsec IKEv1 xAuth protocol in the pfSense operating system, so that VPN clients can connect to the corporate network and start sharing data.

IPsec protocol configuration

Before starting to configure the IPsec IKEv1 xAuth protocol, we must bear in mind that this protocol is oriented to remote access VPN, that is, known as “Mobile Clients”, therefore, the VPN clients will be mobile phones, tablets or computers, but a Site-to-Site tunnel will not be established, to do this it is better to use a protocol such as IPsec IKEv2 directly, without the xAuth that we are going to deal with here.

Configure the «Mobile Clients»

The first thing we have to do to configure the VPN server is to go to the “VPN / IPsec / Mobile Clients” section, we must select the following options:

  • Enable IPsec Mobile Client Support
  • Extended Authentication (xAuth)
    • User authentication: Local Database

In the Extended Authentication section it is also necessary to define the subnet where the VPN clients that connect will be placed.

  • Extended Authentication (xAuth)
    • Virtual Address Pool: provide a virtual IP address to clients, and we put a subnet that is not in use, such as 192.168.100.0/24.
    • DNS Server: provide a DNS Server to clients: here we can put local DNS or public DNS such as Google or Cloudflare.

The final configuration would be as follows:

Once we have saved it, we click on «Save» and above it will put a green button to apply changes, we click on apply changes and when it loads again, we must click on the green button of «Create Phase 1».

We must remember that the IPsec protocol is clearly divided into two phases, phase 1 is the authentication phase, and phase 2 is the phase where symmetric ciphers are negotiated for the exchange of information. First we will have to configure phase 1 with a set of ciphers compatible with most clients, to later configure phase 2.

Configure IPsec Phase 1

In this menu we will have to configure the IPsec protocol correctly to use it with IKEv1 with xAuth, not all the configurations will work with this protocol. Also, a very important detail is that the security settings may change if you use VPN clients for Android, iOS, external programs for Windows, etc. We will use a conservative configuration, compatible with most VPN clients, but you must take this into account because you may have to modify some parameters.

The options that we must configure for it to work correctly are the following:

  • General Information
    • Key Exchange version: IKEv1, if we select any other it will not work.
    • Internet Protocol: IPv4 or IPv6
    • Interface: Internet WAN
    • Description: we put a description.
  • Phase 1 Proposal (Authentication)
    • Authentication Method: Mutual PSK + xAuth
    • Negotiation Mode: Aggresive; selecting “Main” is more secure, but VPN clients may not connect. If the VPN is able to connect well with the configuration we offer, you can later switch to “Main” to see if it works too.
    • My identifier: My IP address
    • Peer identifier: User distinguished name – redeszone@redeszone.net or whatever you want.
    • Pre-Shared Key: 12345678
  • Phase 1 Proposal (Encryption)
    • Encryption Algorithm: AES 128-bit, SHA1, DH Group 2 (1024-bit).

pfSense is a professional operating system, the problem is that VPN clients may not support higher security, for this reason we have not used more robust algorithms such as SHA-512 or a higher DH group. To configure it with maximum security, we must see the IPsec logs and know what “proposals” the VPN clients will send us in order to connect, based on these proposals received, we will be able to choose the most secure of all.

The rest of the configuration options can be left as they come by default.

Once we have configured phase 1, we will get a small summary of what we have just configured, as you can see here:

Once we have configured phase 1 of IPsec xAuth, we are going to configure phase 2.

Configure IPsec Phase 2

In this menu the first thing we must choose is the operating mode, we have chosen “Tunnel IPv4” as you can see in the image. In addition, we will also have to put the «Local Network» that we want the VPN clients to have access, a route will be sent to them to access. We will be able to give them a specific IP address, a subnet that we want, the WAN subnet and even the LAN subnet that we have in pfSense. In the «NAT» option we will leave it at «none».

In this configuration menu we must put the following:

  • General Information
    • Mode: tunnel IPv4
    • Local Network: LAN subnet
    • Description: a description that we want.
  • Phase 2 Proposal (SA / Key Exchange):
    • Protocol: ESP
    • Encryption Algorithm: AES 128-bit and AES-128-GCM 128-bit

In the following photo you can see the final version:

  • Phase 2 Proposal (SA / Key Exchange)
    • Hash algorithms: we select SHA-1 and SHA-256
    • PFS Key group: off, not supported by the protocol.

The rest of the options we can put the default ones, and click on save to save all the changes.

Once finished, we can see the summary of the configuration made.

Now that we have configured the IPsec xAuth VPN server, we have to create the users in pfSense and give them permissions to connect to the VPN, otherwise they will not be able to connect and it will give an error.

Create users in pfSense to authenticate with xAuth

To create the different users, we have to go to ” System / User manager ” and create a new user with its corresponding username and password, click on save, and then re-enter to edit it. Now we can add access permissions to different services and parts of the pfSense operating system. We have to add permission to:

  • Assigned privileges: User – VPN-IPsec xauth Dialin

Once selected, click on «Save» to save the changes. From now on, this user with his corresponding password will be able to connect to the VPN server.

Open ports in the pfSense firewall

In this VPN it is also necessary to open ports on the Internet WAN, specifically we will have to open port 500 UDP and port 4500 UDP. Next, you have all the details to open both ports.

We will have to create a rule in the “Firewall / Rules / WAN” section with the following information:

  • Action: Pass
  • Interface: WAN
  • Address Family: IPv4
  • Protocol: UDP
  • Source: any
  • Destination: WAN Address on port 500

The second rule would be:

  • Action: Pass
  • Interface: WAN
  • Address Family: IPv4
  • Protocol: UDP
  • Source: any
  • Destination: WAN Address on port 4500

As you can see, we have the two rules to accept to allow traffic.

We save and apply changes, ensuring that this rule will be followed. Now we go to the “IPsec” section where we will do a “allow all”. Then when we connect, if we want to limit access, we can do so by putting the corresponding rules here.

  • Action: Pass
  • Interface: IPsec
  • Address Family: IPv4
  • Protocol: any
  • Source: any
  • Destination: any

Next, you can see the newly created rule, which is the only one we have.

Now that we have the IPsec VPN server configured, we have created the user with the corresponding permissions, and we also have it open in the firewall, we are going to perform a connection test with Android.

Connection test

In our case, we have established a VPN connection with an Android smartphone, specifically the Huawei P30 that incorporates an IPsec xAuth PSK client. The configuration that we must carry out is the following (we cannot put capture because the operating system detects it as private content).

  • Name: we give the VPN a name
  • Type: IPsec Xauth PSK
  • Server: IP or DDNS domain of your VPN server
  • IPsec identifier: redeszone@redeszone.net
  • Initial IPsec Shared Key: 12345678; the key we put in the “Pre-Shared Key” section.

We click on save, and connect. At the time of connecting, it will ask us for a username and password, these credentials are the ones that we registered in the operating system itself, and later we gave it permissions to connect correctly. Once done, it will connect us without problems to the VPN server and we will have access to the pfSense administration and any network.

Recommendations and advice

We may have connection problems with some VPN clients if we put encryption algorithms that are too secure, this is completely normal, in addition, depending on the software incorporated in each device, we can configure stronger or weaker algorithms. For security, it is always advisable to choose the most secure ones, but this would prevent many VPN clients from being able to connect.

Whenever an IPsec client is going to connect, it is recommended to view the logs and check what “proposal” the different clients send when connecting. Some smartphones, tablets or IPsec clients support more current encryption, therefore, we should configure the most secure suite possible in such a way that all clients can connect correctly.

With this same tutorial, you will be able to configure the IPsec IKEv1 xAuth, modifying the “Mutual PSK + xAuth” by “Mutual RSA + xAuth” and configuring the corresponding server and client certificates. We will show you how to do it soon. This also brings complications, because if we create a CA with a client certificate that makes use of the latest algorithms, it is possible that it will return an error when we connect because they do not recognize them.