Querying¶

MotorEngine supports a vast array of query operators in MongoDB. There are two ways of querying documents: using the queryset methods (filter, filter_not and the likes) or a Q object.

Querying with filter methods¶

QuerySet.filter(*arguments, **kwargs)

Filters a queryset in order to produce a different set of document from subsequent queries.

Usage:

User.objects.filter(first_name="Bernardo").filter(last_name="Bernardo").find_all(callback=handle_all)
# or
User.objects.filter(first_name="Bernardo", starting_year__gt=2010).find_all(callback=handle_all)

The available filter options are the same as used in MongoEngine.

QuerySet.filter_not(*arguments, **kwargs)¶

Filters a queryset to negate all the filters passed in subsequent queries.

Usage:

User.objects.filter_not(first_name="Bernardo").filter_not(last_name="Bernardo").find_all(callback=handle_all)
# or
User.objects.filter_not(first_name="Bernardo", starting_year__gt=2010).find_all(callback=handle_all)

The available filter options are the same as used in MongoEngine.

Querying with Q¶

class motorengine.query_builder.node.Q(*arguments, **query)¶

A simple query object, used in a query tree to build up more complex query structures.

def handle_users_found(users):
    try:
        assert users[0].name == "Bernardo"
    finally:
        io_loop.stop()

def handle_user_created(user):
    query = Q(name="Bernardo") | Q(age__gt=30)
    users = User.objects.filter(query).find_all(callback=handle_users_found)

def create_user():
    user = User(name="Bernardo", age=32)
    user.save(callback=handle_user_created)

io_loop.add_timeout(1, create_user)
io_loop.start()

The Q object can be combined using python’s binary operators | and &. Do not confuse those with the and and or keywords. Those keywords won’t call the __and__ and __or__ methods in the Q class that are required for the combination of queries.

Let’s look at an example of querying for a more specific document. Say we want to find the user that either has a null date of last update or is active with a date of last_update lesser than 2010:

query = Q(last_update__is_null=True) | (Q(is_active=True) & Q(last_update__lt=datetime(2010, 1, 1, 0, 0, 0)))

query_result = query.to_query(User)

# the resulting query should be similar to:
# {'$or': [{'last_update': None}, {'is_active': True, 'last_update': {'$lt': datetime.datetime(2010, 1, 1, 0, 0)}}]}

assert '$or' in query_result

or_query = query_result['$or']
assert len(or_query) == 2
assert 'last_update' in or_query[0]
assert 'is_active' in or_query[1]
assert 'last_update' in or_query[1]

Query Operators¶

Query operators can be used when specified after a given field, like:

Q(field_name__operator_name=operator_value)

MotorEngine supports the following query operators:

class motorengine.query.exists.ExistsQueryOperator¶

Query operator used to return all documents that have the specified field.

An important reminder is that exists DOES match documents that have the specified field even if that field value is NULL.

For more information on $exists go to http://docs.mongodb.org/manual/reference/operator/query/exists/.

Usage:

class User(Document):
    name = StringField()

query = Q(name__exists=True)

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$exists': True}}

class motorengine.query.greater_than.GreaterThanQueryOperator¶

Query operator used to return all documents that have the specified field with a value greater than the specified value.

For more information on $gt go to http://docs.mongodb.org/manual/reference/operator/query/gt/.

Usage:

class User(Document):
    age = IntField()

query = Q(age__gt=20)

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'age': {'$gt': 20}}

class motorengine.query.greater_than_or_equal.GreaterThanOrEqualQueryOperator¶

Query operator used to return all documents that have the specified field with a value greater than or equal to the specified value.

For more information on $gte go to http://docs.mongodb.org/manual/reference/operator/query/gte/.

Usage:

class User(Document):
    age = IntField()

query = Q(age__gte=21)

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'age': {'$gte': 21}}

class motorengine.query.lesser_than.LesserThanQueryOperator¶

Query operator used to return all documents that have the specified field with a value lower than the specified value.

For more information on $lt go to http://docs.mongodb.org/manual/reference/operator/query/lt/.

Usage:

class User(Document):
    age = IntField()

query = Q(age__lt=20)

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'age': {'$lt': 20}}

class motorengine.query.lesser_than_or_equal.LesserThanOrEqualQueryOperator¶

Query operator used to return all documents that have the specified field with a value lower than or equal to the specified value.

For more information on $lte go to http://docs.mongodb.org/manual/reference/operator/query/lte/.

Usage:

class User(Document):
    age = IntField()

query = Q(age__lte=21)

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'age': {'$lte': 21}}

class motorengine.query.in_operator.InQueryOperator¶

Query operator used to return all documents that have the specified field with a value that match one of the values in the specified range.

If the specified field is a ListField, then at least one of the items in the field must match at least one of the items in the specified range.

For more information on $in go to http://docs.mongodb.org/manual/reference/operator/query/in/.

Usage:

class User(Document):
    age = IntField()

query = Q(age__in=[20, 21, 22, 23, 24])

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'age': {'$in': [20, 21, 22, 23, 24]}}

class motorengine.query.is_null.IsNullQueryOperator¶

Query operator used to return all documents that have the specified field with a null value (or not null if set to False).

This operator uses $exists and $ne for the not null scenario.

For more information on $exists go to http://docs.mongodb.org/manual/reference/operator/query/exists/.

For more information on $ne go to http://docs.mongodb.org/manual/reference/operator/query/ne/.

Usage:

class User(Document):
    email = StringField()

query = Q(email__is_null=False)

query_result = query.to_query(User)

# query results should be like:
# {'email': {'$ne': None, '$exists': True}}

assert 'email' in query_result
assert '$ne' in query_result['email']
assert '$exists' in query_result['email']

class motorengine.query.not_equal.NotEqualQueryOperator¶

Query operator used to return all documents that have the specified field with a value that’s not equal to the specified value.

For more information on $ne go to http://docs.mongodb.org/manual/reference/operator/query/ne/.

Usage:

class User(Document):
    email = StringField()

query = Q(email__ne="heynemann@gmail.com")

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'email': {'$ne': 'heynemann@gmail.com'}}

class motorengine.query.contains.ContainsOperator¶

Query operator used to return all documents which specified field contains a string equal to a passed value.

It is case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__contains='nar')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': 'nar'}}

class motorengine.query.i_contains.IContainsOperator¶

Query operator used to return all documents which specified field contains a string equal to a passed value.

It is not case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__icontains='NaR')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': 'NaR', '$options': 'i'}}

class motorengine.query.ends_with.EndsWithOperator¶

Query operator used to return all documents which specified field ends with passed string value.

It is case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__endswith='do')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': 'do$'}}

class motorengine.query.i_ends_with.IEndsWithOperator¶

Query operator used to return all documents which specified field ends with passed string value.

It is not case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__iendswith='Do')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': 'Do$', '$options': 'i'}}

class motorengine.query.exact.ExactOperator¶

Query operator used to return all documents which specified field is exactly as passed string value.

It is case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__exact='Bernardo')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': '^Bernardo$'}}

class motorengine.query.i_exact.IExactOperator¶

Query operator used to return all documents which specified field is exactly as passed string value.

It is not case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__iexact='bErNaRdO')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': '^bErNaRdO$', '$options': 'i'}}

class motorengine.query.starts_with.StartsWithOperator¶

Query operator used to return all documents which specified field starts with passed string value.

It is case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__startswith='Ber')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': '^Ber'}}

class motorengine.query.i_starts_with.IStartsWithOperator¶

Query operator used to return all documents which specified field starts with passed string value.

It is not case sensitive.

For more information on $regex go to https://docs.mongodb.org/manual/reference/operator/query/regex/

Usage:

class User(Document):
    first_name = StringField()

query = Q(first_name__istartswith='bEr')

query_result = query.to_query(User)

print(query_result)

The resulting query is:

{'name': {'$regex': '^bEr', '$options': 'i'}}

Querying with Raw Queries¶

Even though motorengine strives to provide an interface for queries that makes naming fields and documents transparent, using mongodb raw queries is still supported, both in the filter method and the Q class.

In order to use raw queries, just pass the same object you would use in mongodb:

import tornado.ioloop

class Address(Document):
    __collection__ = "QueryingWithRawQueryAddress"
    street = StringField()

class User(Document):
    __collection__ = "QueryingWithRawQueryUser"
    addresses = ListField(EmbeddedDocumentField(Address))
    name = StringField()

def create_user():
    user = User(name="Bernardo", addresses=[Address(street="Infinite Loop")])
    user.save(callback=handle_user_created)

def handle_user_created(user):
    User.objects.filter({
        "addresses": {
            "street": "Infinite Loop"
        }
    }).find_all(callback=handle_find_user)

def handle_find_user(users):
    try:
        assert users[0].name == "Bernardo", users
        assert users[0].addresses[0].street == "Infinite Loop", users
    finally:
        io_loop.stop()

io_loop.add_timeout(1, create_user)
io_loop.start()

Retrieving a subset of fields¶

Sometimes a subset of fields on a Document is required, and for efficiency only these should be retrieved from the database. There are some methods that could be used to specify which fields to retrieve. Note that if fields that are not downloaded are accessed, their default value (or None if no default value is provided) will be given.

Projections for reference fields (and a list of reference fields) can be specified too in the same way as for embedded fields. They are applied immediately if lazy is False or will be applied later after .load_reference() will be called otherwise.

Note

You can use BlogPost.title notation instead of string value ‘title’ only for the first level fields. So BlogPost.author.name will not work, use string ‘author.name’ instead. Also _id field should be always specified as string ‘_id’.

Note

It is not possible to save document with projection specified during retrieving. It will raise exception motorengine.errors.PartlyLoadedDocumentError as you would try that.

QuerySet.only(*fields)¶

Load only a subset of this document’s fields.

Usage:

BlogPost.objects.only(BlogPost.title, "author.name").find_all(...)

Note

only() is chainable and will perform a union. So with the following it will fetch both: title and comments:

BlogPost.objects.only("title").only("comments").get(...)

Note

only() does not exclude _id field

all_fields() will reset any field filters.

Parameters:	fields – fields to include

QuerySet.exclude(*fields)¶

Opposite to .only(), exclude some document’s fields.

Usage:

BlogPost.objects.exclude("_id", "comments").get(...)

Note

exclude() is chainable and will perform a union. So with the following it will exclude both: title and author.name:

BlogPost.objects.exclude(BlogPost.title).exclude("author.name").get(...)

Note

if only() is called somewhere in chain then exclude() calls remove fields from the lists of fields specified by only() calls:

# this will load all fields
BlogPost.objects.only('title').exclude('title').find_all(...)

# this will load only 'title' field
BlogPost.objects.only('title').exclude('comments').get(...)

# this will load only 'title' field
BlogPost.objects.exclude('comments').only('title', 'comments').get(...)

# there is one exception for _id field,
# which will be excluded even if only() is called,
# actually the following is the only way to exclude _id field
BlogPost.objects.only('title').exclude('_id').find_all(...)

all_fields() will reset any field filters.

Parameters:	fields – fields to exclude

QuerySet.all_fields()¶

Include all fields.

Reset all previously calls of .only() or .exclude().

Usage:

# this will load 'comments' too
BlogPost.objects.exclude("comments").all_fields().get(...)

QuerySet.fields(_only_called=False, **kwargs)¶

Manipulate how you load this document’s fields.

Used by .only() and .exclude() to manipulate which fields to retrieve. Fields also allows for a greater level of control for example:

Retrieving a Subrange of Array Elements:

You can use the $slice operator to retrieve a subrange of elements in an array. For example to get the first 5 comments:

BlogPost.objects.fields(slice__comments=5).get(...)

or 5 comments after skipping 10 comments:

BlogPost.objects.fields(slice__comments=(10, 5)).get(...)

or you can also use negative values, for example skip 10 comment from the end and retrieve 5 comments forward:

BlogPost.objects.fields(slice__comments=(-10, 5)).get(...)

Besides slice, it is possible to include or exclude fields (but it is strongly recommended to use .only() and .exclude() methods instead):

BlogPost.objects.fields(
    slice__comments=5,
    _id=QueryFieldList.EXCLUDE,
    title=QueryFieldList.ONLY
).get(...)

Parameters:	kwargs – A dictionary identifying what to include