The Price of all “Rest with Spring” course packages will increase by $50 next Friday:

>>> GET ACCESS NOW

1. Overview

In this article, we’ll extend the REST Query Language we developed in the previous parts of the series to include more search operations.

We now support the following operations: Equality, Negation, Greater than, Less than, Starts with, Ends with, Contains and Like.

Note that we explored three implementations – JPA Criteria, Spring Data JPA Specifications and Query DSL; we’re going forward with Specifications in this article because it’s a clean and flexible way to represent our operations.

2. The SearchOperation enum

First – let’s start by defining a better representation of our various supported search operations – via an enumeration:

public enum SearchOperation {
    EQUALITY, NEGATION, GREATER_THAN, LESS_THAN, LIKE, STARTS_WITH, ENDS_WITH, CONTAINS;

    public static final String[] SIMPLE_OPERATION_SET = { ":", "!", ">", "<", "~" };

    public static SearchOperation getSimpleOperation(char input) {
        switch (input) {
        case ':':
            return EQUALITY;
        case '!':
            return NEGATION;
        case '>':
            return GREATER_THAN;
        case '<':
            return LESS_THAN;
        case '~':
            return LIKE;
        default:
            return null;
        }
    }
}

We have two sets of operations:

1. Simple – can be represented by one character:

  • Equality: represented by colon (:)
  • Negation: represented by Exclamation mark (!)
  • Greater than: represented by (>)
  • Less than: represented by (<)
  • Like: represented by tilde (~)

2. Complex – need more than one character to be represented:

  • Starts with: represented by (=prefix*)
  • Ends with: represented by (=*suffix)
  • Contains: represented by (=*substring*)

We also need to modify our SearchCriteria class to use the new SearchOperation:

public class SearchCriteria {
    private String key;
    private SearchOperation operation;
    private Object value;
}

3. Modify UserSpecification

Now – let’s include the newly supported operations into our UserSpecification implementation:

public class UserSpecification implements Specification<User> {

    private SearchCriteria criteria;

    @Override
    public Predicate toPredicate(
      Root<User> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
    
        switch (criteria.getOperation()) {
        case EQUALITY:
            return builder.equal(root.get(criteria.getKey()), criteria.getValue());
        case NEGATION:
            return builder.notEqual(root.get(criteria.getKey()), criteria.getValue());
        case GREATER_THAN:
            return builder.greaterThan(root.<String> get(
              criteria.getKey()), criteria.getValue().toString());
        case LESS_THAN:
            return builder.lessThan(root.<String> get(
              criteria.getKey()), criteria.getValue().toString());
        case LIKE:
            return builder.like(root.<String> get(
              criteria.getKey()), criteria.getValue().toString());
        case STARTS_WITH:
            return builder.like(root.<String> get(criteria.getKey()), criteria.getValue() + "%");
        case ENDS_WITH:
            return builder.like(root.<String> get(criteria.getKey()), "%" + criteria.getValue());
        case CONTAINS:
            return builder.like(root.<String> get(
              criteria.getKey()), "%" + criteria.getValue() + "%");
        default:
            return null;
        }
    }
}

4. Persistence Tests

Next – we let’s test our new search operations – at the persistence level:

4.1. Test Equality

In the following example – we’ll search for a user by their first and last name:

@Test
public void givenFirstAndLastName_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.EQUALITY, "john"));
    UserSpecification spec1 = new UserSpecification(
      new SearchCriteria("lastName", SearchOperation.EQUALITY, "doe"));
    List<User> results = repository.findAll(Specifications.where(spec).and(spec1));

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.2. Test Negation

Next, let’s search for users that by the their first name not “john”:

@Test
public void givenFirstNameInverse_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.NEGATION, "john"));
    List<User> results = repository.findAll(Specifications.where(spec));

    assertThat(userTom, isIn(results));
    assertThat(userJohn, not(isIn(results)));
}

4.3. Test Greater Than

Next – we will search for users with age greater than “25”:

@Test
public void givenMinAge_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("age", SearchOperation.GREATER_THAN, "25"));
    List<User> results = repository.findAll(Specifications.where(spec));

    assertThat(userTom, isIn(results));
    assertThat(userJohn, not(isIn(results)));
}

4.4. Test Starts With

Next – users with their first name starting with “jo”:

@Test
public void givenFirstNamePrefix_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.STARTS_WITH, "jo"));
    List<User> results = repository.findAll(spec);

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.5. Test Ends With

Next we’ll search for users with their first name ending with “n”:

@Test
public void givenFirstNameSuffix_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.ENDS_WITH, "n"));
    List<User> results = repository.findAll(spec);

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.6. Test Contains

Now, we’ll search for users with their first name containing “oh”:

@Test
public void givenFirstNameSubstring_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.CONTAINS, "oh"));
    List<User> results = repository.findAll(spec);

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.7. Test Range

Finally, we’ll search for users with ages between “20” and “25”:

@Test
public void givenAgeRange_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("age", SearchOperation.GREATER_THAN, "20"));
    UserSpecification spec1 = new UserSpecification(
      new SearchCriteria("age", SearchOperation.LESS_THAN, "25"));
    List<User> results = repository.findAll(Specifications.where(spec).and(spec1));

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

5. The UserSpecificationBuilder

Now that persistence is done and tested, let’s move our attention to the web layer.

We’ll build on top of the UserSpecificationBuilder implementation from the previous article to incorporate the new new search operations:

public class UserSpecificationsBuilder {

    private List<SearchCriteria> params;

    public UserSpecificationsBuilder with(
      String key, String operation, Object value, String prefix, String suffix) {
    
        SearchOperation op = SearchOperation.getSimpleOperation(operation.charAt(0));
        if (op != null) {
            if (op == SearchOperation.EQUALITY) {
                boolean startWithAsterisk = prefix.contains("*");
                boolean endWithAsterisk = suffix.contains("*");

                if (startWithAsterisk && endWithAsterisk) {
                    op = SearchOperation.CONTAINS;
                } else if (startWithAsterisk) {
                    op = SearchOperation.ENDS_WITH;
                } else if (endWithAsterisk) {
                    op = SearchOperation.STARTS_WITH;
                }
            }
            params.add(new SearchCriteria(key, op, value));
        }
        return this;
    }

    public Specification<User> build() {
        if (params.size() == 0) {
            return null;
        }

        List<Specification<User>> specs = new ArrayList<Specification<User>>();
        for (SearchCriteria param : params) {
            specs.add(new UserSpecification(param));
        }

        Specification<User> result = specs.get(0);
        for (int i = 1; i < specs.size(); i++) {
            result = Specifications.where(result).and(specs.get(i));
        }
        return result;
    }
}

6. The UserController

Next – we need to modify our UserController to correctly parse the new operations:

@RequestMapping(method = RequestMethod.GET, value = "/users")
@ResponseBody
public List<User> findAllBySpecification(@RequestParam(value = "search") String search) {
    UserSpecificationsBuilder builder = new UserSpecificationsBuilder();
    String operationSetExper = Joiner.on("|").join(SearchOperation.SIMPLE_OPERATION_SET);
    Pattern pattern = Pattern.compile(
      "(\w+?)(" + operationSetExper + ")(\p{Punct}?)(\w+?)(\p{Punct}?),");
    Matcher matcher = pattern.matcher(search + ",");
    while (matcher.find()) {
        builder.with(
          matcher.group(1), 
          matcher.group(2), 
          matcher.group(4), 
          matcher.group(3), 
          matcher.group(5));
    }

    Specification<User> spec = builder.build();
    return dao.findAll(spec);
}

We can now hit the API and get back the right results with any combination of criteria. For example – here’s a what a complex operation would look like using API with the query language:

http://localhost:8080/users?search=firstName:jo*,age<25

And the response:

[{
    "id":1,
    "firstName":"john",
    "lastName":"doe",
    "email":"[email protected]",
    "age":24
}]

7. Tests for the Search API

Finally – let’s make sure our API works well by writing a suite of API tests.

We’ll start with the simple configuration of the test and the data initialization:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(
  classes = { ConfigTest.class, PersistenceConfig.class }, 
  loader = AnnotationConfigContextLoader.class)
@ActiveProfiles("test")
public class JPASpecificationLiveTest {

    @Autowired
    private UserRepository repository;

    private User userJohn;
    private User userTom;

    private final String URL_PREFIX = "http://localhost:8080/users?search=";

    @Before
    public void init() {
        userJohn = new User();
        userJohn.setFirstName("John");
        userJohn.setLastName("Doe");
        userJohn.setEmail("[email protected]");
        userJohn.setAge(22);
        repository.save(userJohn);

        userTom = new User();
        userTom.setFirstName("Tom");
        userTom.setLastName("Doe");
        userTom.setEmail("[email protected]");
        userTom.setAge(26);
        repository.save(userTom);
    }

    private RequestSpecification givenAuth() {
        return RestAssured.given().auth()
                                  .preemptive()
                                  .basic("username", "password");
    }
}

7.1. Test Equality

First – let’s search for a user with the first name “john” and last name “doe:

@Test
public void givenFirstAndLastName_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:john,lastName:doe");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.2. Test Negation

Now – we’ll search for users when their first name isn’t “john”:

@Test
public void givenFirstNameInverse_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName!john");
    String result = response.body().asString();

    assertTrue(result.contains(userTom.getEmail()));
    assertFalse(result.contains(userJohn.getEmail()));
}

7.3. Test Greater Than

Next – we will look for users with age greater than “25”:

@Test
public void givenMinAge_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "age>25");
    String result = response.body().asString();

    assertTrue(result.contains(userTom.getEmail()));
    assertFalse(result.contains(userJohn.getEmail()));
}

7.4. Test Starts With

Next – users with their first name starting with “jo”:

@Test
public void givenFirstNamePrefix_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:jo*");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.5. Test Ends With

Now – users with their first name ending with “n”:

@Test
public void givenFirstNameSuffix_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:*n");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.6. Test Contains

Next, we’ll search for users with their first name containing “oh”:

@Test
public void givenFirstNameSubstring_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:*oh*");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.7. Test Range

Finally, we’ll search for users with ages between “20” and “25”:

@Test
public void givenAgeRange_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "age>20,age<25");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

8. Conclusion

In this article we brought the query language of our REST Search API forward to a mature, tested, production grade implementation. We now support a wide variety of operations and constraints, which should make it quite easy to cut across any dataset elegantly and get to the exact resources we’re looking for.

The full implementation of this article can be found in the github project – this is an Eclipse based project, so it should be easy to import and run as it is.

The Price of all “Rest with Spring” course packages will increase by $50 next Friday:

>>> GET ACCESS NOW

Sort by:   newest | oldest | most voted
Pete
Guest

Nicely done;
Will this work with DateTime data type?

Eugen Paraschiv
Guest

Hey Pete – these particular operations here don’t make use of dates, but – sure – no reason you can’t just implement a bit of logic to handle dates as well. As you can see, the operations are quite flexible – you can add anything you can then implement in the persistence layer quite easily. Cheers,
Eugen.

Ravindra
Guest

Hi Eugen, How to implement this search API with relational tables?

Eugen Paraschiv
Guest

Hey Ravindra – this entire series is all backed by standard SQL. The series covers JPA Criteria, JPA Specifications and QueryDSL, which are all abstractions on top of SQL – so any of these should be OK. Hope that helps. Cheers,
Eugen.

Yochi
Guest

Hi Eugen, What gonna happen when search keyword contains with the space ?

Eugen Paraschiv
Guest

It depends – as long as that gets to to the back-end correctly (properly encoded) it should be OK.

Luboš Svoboda
Guest

The regex pattern used in this example allows only word characters to be passed as value of the criteria – I think this is the biggest issue of this solution. Replacing the pattern (\w+?) with (.*) will fail with parameter containing comma character. How to solve it? Encoding parameter value to base64?

Eugen Paraschiv
Guest
That’s an interesting point – here are a couple of notes on that. First – the example here is certainly not production ready in terms of flexibility – it’s a base on which you can build on. Second – encoding is an option, sure – or replacing the regex with something more open (but not as open as your alternative of course) would be another option. But, point number one is what’s important here – there are several things I would build into the implementation before going to prod with it, and this is one of those things. Hope that… Read more »
Jim Kerak
Guest

Does this approach work if I need to search a list of entities based on the value of a related entity? (example, find all users with a certain users.address.city). I need to have optional query parameters that, if present, join to related tables and be included in the where clause. I have classically done this is SQL with stored procedures by having the where clause look something like this where @UserID and @City are nullable input params:

WHERE (@UserID IS NULL OR User.UserID = @UserID)
AND (@City IS NULL or Address.City = @City)

Eugen Paraschiv
Guest

Hey Jim,
That’s an interesting question, definitely.
The thing to understand with this approach is that it give you the building blocks to implement the queries you need.
You’re certainly not limited to these operations here – and can of course build your own.
And so, in your case, sure – the syntax should support your scenario perfectly fine – and then of course you’ll have to hook that up to your persistence layer.
Hope that helps point you in the right direction. Cheers,
Eugen.

Jim Kerak
Guest

I found that I was successfully able to implement the desired approach, but the quantity of “building blocks” was getting too great for the database schema I am working with. So for me, it made sense to define my queries in HQL (@Query annotation on JpaRepository), and for particularly complex queries or batch updates, to use stored procedures.

Eugen Paraschiv
Guest

Yeah, there’s definitely a balance here.
That’s actually why I covered more than one approach in this series – using different types of persistence layers. You’ll find that each usecase fits one of these better – and that will help keep the complexity relatively under control. Of course there’s inherent complexity in the problem itself.
I’m glad you were able to get it to work though. Cheers,
Eugen.

wpDiscuz