spatialed

Help Center: to get (image) statistics

2008-08-28T22:47:07.0005Z

The following is the code I used. Have a look only if you're free. I print out $count to see number of topics with images and $count~438

Afterwards I grep entire query result and got numbers close to you:

692001 topics with image

626805 topics with 1 image

62827 topics with 2 images

2210 topcis with 3 images

and so on...

#!/usr/bin/perl
use URI::Escape;
# This module provides the uri_escape function used below

# Build the Metaweb query, using string manipulation
# CAUTION: the use of string manipulation here makes this script vulnerable
# to MQL injection attacks when the command-line argument includes JSON.
#$band = $ARGV[0]; # This is the band or musician whose albums are to be listed

my $cursor = 'true';
my $query;
my $pre =
'{
    "qname":{
        "cursor":';

my $post = ',
        "query":[{
            "id":null,
            "limit":1000,
            "type":"/common/topic",
            "image":[{
                "return":"count",
                "id":null
            }]
        }]
    }
}';

my $allresult = '';
my $count = 0;
my $result = '';

until($cursor =~ /error/){
#while($cursor){
    $query = $pre . $cursor . $post;

    $escaped = uri_escape($query);

    $baseurl='http://www.freebase.com/api/service/mqlread'; # Base URL for queries
    $url = $baseurl . "?queries=" . $escaped;

    $auth = 'metaweb-user=###Enter Your cookie data here###';

    #$result = `curl -s --cookie \'$auth\' $url`;
    $result = `curl -s $url`;


    $result =~ /\/api\/status\/(.*)/;
    $status = $1;

    $result =~ /cursor\":\s*(\".*\")/;
    $cursor = $1;

    $count = $count + 1;

    print $result . "\n";
}

# Use regular expressions to extract the album list from the HTTP response
#$result =~ s/^.*"album"\s*:\s*\[\s*([^\]]*)\].*$/$1/s;
#$result =~ s/[ \t]*"[ \t,]*//g;
#print $allresult ."\n";

print 'done'."\n";
# Finally, display the list of albums
#print "$result\n";
#
# vim: ts=2 sw=2

Help Center: to get (image) statistics

2008-08-28T20:34:47.0012Z

Well, I got essentially the same results as reported above using metaweb.py. Are you using your own Perl module, or one from our API library?

Help Center: to get (image) statistics

2008-08-28T15:19:55.0012Z

Thanks. I counted the number of iteration for the code above and multiply it by 1000 (since I set the limit 1000) to get approximate image count.

Help Center: to get (image) statistics

2008-08-28T05:53:50.0012Z

Some MQL reference material can be found here. I iterated through topics with images and then recorded the number of links there were. Currently, you can't constrain the query to ask it to return topics with exactly n amount of images. I'm not sure why your results differ so greatly that mine - I'll try to investigate a little more.

Help Center: to get (image) statistics

2008-08-28T05:04:39.0012Z

Thank you. It seems I don't know enough of MQL language; let me know if there's reference other than this website for I don't want to bother you like this.

I use the following code embedded in a perl code to count the number of topics with at least one image, and the result is about 438K--much lower than yours. By the way how to get the number of topics with exactly 1, 2, or 3 images? Can this be requested directly in the query?

{
"qname":{
"cursor":<cursor value>

    "query":[{
      "id":null,
      "limit":1000,
      "type":"/common/topic",
      "image":[{
        "id":null
      }]
    }]
}
}

Help Center: to get (image) statistics

2008-08-27T22:31:31.0012Z

You can just break the link between an image and a topic and not have the image deleted. Just for fun, I ran a quick analysis:

Found 716008 /common/topic topics with image links (out of ~4 million topics)
650326 have one img
63058 have two imgs
2296 have three imgs
328 have four or more imgs

Note that more than one topic can be linked to the same image.

Help Center: to get (image) statistics

2008-08-27T20:54:54.0012Z

Thank you. Then if a /common/image object is not linked to any /common/topic object, why is it there and how can it be presented to end users? Thanks.

Help Center: to get (image) statistics

2008-08-27T19:06:31.0012Z

No, not necessarily - a /common/image topic may not be linked to a /common/topic topic (for whatever reason). You could use the last query to find out how many images each /common/topic topic has by counting the ids in the "image" clause.

Help Center: to get (image) statistics

2008-08-27T17:58:20.0012Z

Thanks, that's really helpful!

Now if i'm not mistaken there're about 700K images and 330K topics with image. Does that imply that on average those topics with images each has 2 images? Thanks.

Organism Classification: Conservation Status

2008-08-26T21:57:06.0012Z

IUCN Red List http://www.iucnredlist.org/search/search-basic

This site has a lots of useful info.

Help Center: to get (image) statistics

2008-08-26T16:34:25.0000Z

The result times out because there is a large amount of /common/image topics. Usually, the way to request counts is:

{
"id" : null,
"return" : "count",
"type" : "/common/image"
}

But this will still time out on requests with a large amount of topics. You could query for the instance count which is tallied nightly:

{
"/freebase/type_profile/instance_count" : null,
"id" : "/common/image"
}

but that isn't real-time.

Lastly, you could count yourself by using a cursor and counting until the end of the result set:

{
"cursor": true,
    "query": [{
       "id" : null,
     "type" : "/common/image"
    }]
}

To find topics that have an image, you can query for them like so:

[
{
    "id" : null,
    "image" : [
      {
        "id" : null
      }
    ],
    "type" : "/common/topic"
}
]

Again, the resultset is very large and will time out, so you will need to cursor and count yourself.

Data Modeling Guide: Duplicate values

2008-08-26T13:15:38.0012Z

There is not, because the way you’ve structured your query, you are asking for the “acting gigs,” not the actors. It appears that Tambor did play two different roles on Arrested Development, and so two gigs are appropriate there. If you ask for the character in the cast, this will be more obvious.

Data Modeling Guide: Duplicate values

2008-08-26T07:51:14.0005Z

Hi,

I send this query:

[
{
    "name" : null,
    "original_network" : [
      {
        "network" : null
      }
    ],
    "regular_cast" : [
      {
        "actor" : null
      }
    ],
    "type" : "/tv/tv_program"
}
]

and i get an actor "Jefferey Tambor" that appear twice in "Arrested Development".

Is there anyway to ignore from duplicate values?

Tx All.

Medicine: suggested properties

2008-08-26T00:10:44.0028Z

mechanism of action

adult dosing

child dosing

Protected Places: Some protected area databases

2008-08-24T16:52:43.0012Z

The Jamaica Protected Areas Trust is creating a web site with interactive virtual tour maps linking to protected area location 360 degree panorama views with audio, detailed information, and a Conservation Network Center for public involvement.

The web site also contains a link to comprehensive GIS datasets on Jamaica. See The Nature Conservancy Internet Map Services.

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T18:09:02.0012Z

Bear in mind also that ids must be unique within a namespace. If catalog numbers are all unique, this isn't a problem. But if they're only unique per label, this might be enormously cumbersome. If I'm reading you right, Chris, I'm seeing a one-namespace-per-label model, which, while theoretically do-able, seems like a vast undertaking.

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T16:54:33.0012Z

You can make namespaces, such as /user/metaeducation/labels and /user/metaeducation/labels/capitol, with the query editor. Create an instance of /type/namespace and add a key into the parent namespace.

{

"type":"/type/namespace",

"name":"My Namespace",

"key":{

"value":"sample",

"namespace":"/user/metaeducation"

"create":"unless_exists"

}

will create an instance of Namespace addressable as /user/metaeducation/sample. You can add things to that namespace:

{

"id":"/guid/deadbeefcafebabedeadbabecafebeef",

"key":{

"value":"why_dead_beef",

"namespace":"/user/metaeducation/sample",

"connect":"insert"

}

As for uniqueness of albums: the problem is that MusicBrainz only models releases, not actual albums. We attempt to identify releases of the same albums, but we inevitably miss some as it is an automated process. Duplicate albums should be marked for merge.

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T14:09:59.0012Z

Really? This sounds good. How would I go about doing this?

As you probably guessed from our discussion in Music, I'm actually trying to link to albums by catalog #. (Imagine a manufacturer who has several releases of the same album, under different numbers, and would like the links to go to the appropriate Freebase article. So that's what we're looking for. A pretty URL with the catalog number in it, instead of GUID, which takes one to the appropriate page to upload cover art or edit the track info.)

It appears, however, that the albums are not correctly organized right now, since multiple releases of the same album are in Freebase as separate "albums". I'm worried about how doing the work of properly sorting releases might interfere with synchronizing the data with MusicBrainz... is it better just to let well enough alone?

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T13:05:08.0012Z

We’ve done this for some properties already, like NAICS industry classification, IMDb IDs for actors and films. You can do it yourself in your own namespace; it might be a bit unwieldy (/user/metaeducation/isbn/0817525766) but it would work. If you come up with a good one we could link it into a shorter place (like /authority/isbn).

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T02:51:45.0012Z

Hmmm...

Well, if I could spec this, I'd like to be able to form an inbound link to Freebase by providing a property name and a value, and simply get a page to view the article from that. e.g. ("book/book_edition/isbn", "817525766") If this does not specify a unique article, then come up with a list of all matches.

It would be trivial for Metaweb to add an API that did this, it's just that once you start putting question marks and such in things it starts to confuse users, e.g.

http://freebase.com/api/view?property="book/book_edition/isbn"?value="817525766"

So maybe overloading view (or something otherwise prettier) could be good:

http://freebase.com/view/book/book_edition/isbn=817525766

A key goal would be that when you loaded an article this way, you would not be redirected, but rather see that "pretty" URL in the address bar.

This way pretty links would come from properties of the data, without anyone having to do extra work in disambiguation that hadn't already been done.

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T01:45:27.0012Z

I think freebase has mimicked wikipedia in populating the "/view/en" (i.e. english language) namespace. Clearly, there can be only one leonardo_da_vinci, and that's not a problem in many cases. There are probably many topics in freebase that could be reliably identified by their common name in that space but the freebase staff (who control it) haven't caught up. Perhaps some collaborative feature in this are would be helpful e.g. "suggest a unique name for this topic", "flag this topic as more deserving of 'john_smith' " :)

your idea of creating other url paths that represent namespaces is fantastic e.g. NYSE stock symbols could be /view/nyse/ge. This could go a long way for providing user readable urls to topics.

I think freebase.com should consider this a pretty hot feature request; it's not possible now. At least, the company could start creating url/namespaces, but allowing users to create (or at least nominate them) should be considered as well.

Help Center: Prettier links to Freebase than GUIDs

2008-08-22T00:21:44.0017Z

Hi there...

I'm trying to use a "prettier" URL in my application than to the Freebase GUID-based one. Got my hopes up from what I read at http://www.readwriteweb.com/archives/freebase_overview.php

"Each permanent object in the system has a GUID - a unique identifier, something like this: #9202a8c040000064..... The identifier can be used to refer to the object via URL and via queries. In addition to the GUID, there are other ways to refer to the object, for example, http://www.freebase.com/view/en/leonardo_da_vinci. Beyond that, there are even other aliases, for example, you can refer to a public company by its stock ticker symbol. But regardless of the reference, the key point is that you end up with the same, unique node in the system."

That sounded good, but... how can you do this, besides typing in the stock symbol as an "alias"?

I'd imagine it's very common for a domain to have a property which is generally unique within that space. All things being equal, people would like to see a link with that instead of the GUID. But to have to type it in twice (once as a property and once as an "alias") seems like the wrong answer..

There are a lot of cool things in api/* but I haven't found one that would finesse something like this, any pointers? I'm just trying to get a link that will jump to the Freebase page for the topic...based on, say, the ISBN# of a book.

Roadmap of future features: How fast is freebase growing?

2008-08-21T02:40:35.0012Z

Well, we don't have all of these reports, but you can take a look at what we do have here... Reporting Tools for Freebase Experts

We hope to add more such tools in the future. Thanks for your interest and suggestions!

BMW: BMW manufacturer vs. automobile or motorcycle make

2008-08-19T23:42:55.0012Z

Ah, right, separating the make from the company. I get it now. And it makes a lot of sense.

BMW: BMW vs BMW Group

2008-08-19T21:43:00.0012Z

This makes sense to me. The "Make" type doesn't really belong here; the brands can be brought in via the Automobile Company/Makes property. (And it looks like we actually need a date-mediator on the Makes property, since MINI and Land Rover have been makes of different companies over the years.

BMW: BMW manufacturer vs. automobile or motorcycle make

2008-08-19T21:37:56.0012Z

I agree with Ed -- the topic he links to ("BMW") should be reserved for the makes (I don't know whether the automobile and motorcycle makes should be different topics), and this topic should be reserved for whichever part of the BMW Group it corresponds to. But the two, company and make, should not use the same types.

Roadmap of future features: How fast is freebase growing?

2008-08-19T19:23:46.0005Z

Hello,

It would be nice to have a page with some number showing how fast the database is growing.

Like:

New releations added in the last 24 hours

Releations deleted in the last 24 hours

Topics created in the last 24 hours

New releations added in the last 7 days

Releations deleted in the last 7 days

Topics created in the last 7 days

...

Or may be such a page already exists and I just don't know?

- Ralf

BMW: BMW vs BMW Group

2008-08-19T19:15:52.0012Z

I'm sorry, I checked my source again and there is a subsidiary for Rolls-Royce Motor Cars GmbH in Germany and Rolls-Royce Motor Cars Ltd in England.

However, I think it is safe to say that MINI, Rolls-Royce and BMW are brands of the company.

Actual legal subsidiaries are often a totally different thing. For BMW's "principal subsidiaries", you can check their Annual Report.

BMW: BMW manufacturer vs. automobile or motorcycle make

2008-08-19T19:08:38.0012Z

This topic appears to be for the parent company Bayerische Motoren Werke AG, so we need to make sure the integrity of the company remains if we break it up.

BMW: BMW vs BMW Group

2008-08-19T19:06:49.0012Z

Bayerische Motoren Werke AG is the parent company and it has many BMW subsidiaries (like BMW Japan Corp, BMW Canada Inc).

Rolls Royce is a brand, not a subsidiary (and it is not to be confused with the Rolls-Royce Group Plc!). Mini is also a brand, not a subsidiary.

Data Modeling Guide: New data

2008-08-19T05:12:35.0012Z

Hi,

Thanks a lot for your help.

Data Modeling Guide: New data

2008-08-07T17:19:48.0012Z

To be precise, Bryan’s query will find things that were created this month, which are also of type /film/film. To find new assertions, you have to get a little more complicated.

[
{
    "id" : null,
    "type" : {
      "id" : "/film/film",
      "link" : {
        "timestamp>" : "2008-08"
      }
    }
}
]

will find things who have had their film type asserted this month.

[
{
    "id" : null,
    "type" : [
      {
        "id" : null,
        "link" : {
          "timestamp>" : "2008-08"
        }
      }
    ]
}
]

will find things that have had new types added this month. In general, for any property assertion, you can ask about the timestamp on the link.

Data Modeling Guide: New data

2008-08-07T16:29:24.0012Z

You can query against the timestamp, something along the lines of:

[
{
    "name" : null,
    "timestamp" : null,
    "timestamp>" : "2008-08",
    "type" : "/film/film"
}
]

This returns films that were added this month.

Data Modeling Guide: New data

2008-08-07T04:30:18.0017Z

Hi,

How can i get the data that were added from specific date (mqlread)

Tx.

spatialed: Properties co-typed as other types

2008-07-01T15:34:09.0005Z

Thanks, Ed.
I think Physiognomy and Satellite sensor status (the properties) should be separate, discrete objects than Physiognomy and Satellite sensor status (the topic and LC) and not conflated. You do have a separate Physiognomy type (http://www.freebase.com/tools/schema/user/spatialed/land_cover/physiognomy) - should that have included types of topic and LC?

spatialed: Properties co-typed as other types

2008-07-01T04:09:05.0012Z

Sure but they were purposeful typings. LC is a bucket type I created for all types that have something to do with land cover mapping even if they are best placed in another domain. They are usually topics too. No problem if you want to detype them. I've put that project on the shelf for a little bit and know how to pick it back up later.

spatialed: Properties co-typed as other types

2008-06-30T16:41:39.0012Z

Hi Ed,

I've noticed that you have 2 properties, Physiognomy and Satellite sensor status cotyped as LC and Topic. I'm guessing this might have been an errant mistyping due to autocomplete. I'd like to detype them - is that ok?

Organism Classification: Conservation Status

2008-06-11T20:37:42.0001Z

I happened to be looking at my state's endangered & threatened wildlife list and they had both state and federal status.

So that might be something to consider. Even if you only want to do federal level, it should be clear that it is.

And something else I just thought of, is you'd probably need a country field, or source/location, so people knew where it was endangered/threatened.

Organism Classification: Conservation Status

2008-06-11T20:01:33.0000Z

I agree with Ed's suggestion about the model, but I would add a date for delisting as well.

What I'd recommend is creating an "organism classification" type (or something similarly-named) in your private domain with the "conservation status" property, and try out some models there. Post back here (or on the data-modelers' list) when/if you want feedback. If it seems to work well, it could eventually get moved to the commons Biology domain.

Organism Classification: Conservation Status

2008-06-11T05:41:15.0007Z

I'd suggest modeling an "organism conservation status" type or something more inclusive sounding and then add it on a case by case basis. For example, I have imported Partners in Flight conservation status codes that are only applicable to birds in the Western Hemisphere. These should probably be added as part of CVTs with date and location properties. You could also add US conservation categories of threatened and endangered or subset it for state listings.

I would create a CVT with status code (that links to the status classification system), date, and location at a minimum. For example, Bald Eagles are going through a US federal delisting period but may still be listed as endangered at the state level in some states and their status today in the US is different than it was a year ago. So, those three properties (code, date, location) should be sufficient but are also essential.

Organism Classification: Conservation Status

2008-06-11T04:28:19.0005Z

I'd like to add a "conservation status" property to the schema, so that I can start adding endangered/extinct species data.

I'm assuming that "organism classification" would be the right topic...

Protected Places: Some protected area databases

2008-05-05T21:49:16.0000Z

I found the copyright info for UNEP-WCMC data (the last link in my initial post): "Use and reproduction of this data is authorised for educational or other non-commercial purposes without prior permission from the copyright holders. Use or reproduction of the data for commercial purposes is prohibited without the prior written permission of the copyright holders."

Protected Places: Some protected area databases

2008-05-05T21:48:49.0018Z

WDPA is strictly non-commercial. The protected areas database is selling their data, which doesn't usually map well to CC-BY. I can't find anything about IUCN.

Protected Places: Some protected area databases

2008-05-05T21:30:25.0007Z

Here's another one: Managed Areas Database

Note: "RSRU no longer staffs resources able to update this dataset. We are interested in finding a responsible agency to take over the update/upkeep process for the MAD database. Since MAD has been described as "incredibly valuable" by many of the education and research community, it is hoped that someone would be willing to take over."

Protected Places: Some protected area databases

2008-05-05T21:01:46.0007Z

No idea. Those funded through US federal grants are supposed to be open access but that often is not the case. I would expect the same goes for databases funded by other governments.

Protected Places: Some protected area databases

2008-05-05T19:41:04.0007Z

Thanks! Do you know what the licensing on those databases is?

Protected Places: Some protected area databases

2008-05-04T17:35:46.0005Z

World Database on Protected Areas

Protected Area Database

IUCN Protected Area Program and related Prototype Nationally Designated Protected Areas Database

spatialed: Cyclical included types

2008-04-17T03:03:35.0000Z

Good catch. Thanks. I was still figuring out how to work with Included types when I created those schema. They probably still need some work but I removed the cyclical relationships. Let me know if you see any other problems. I plan to get back to these schema and start adding a lot more data for them May-June.

spatialed: Cyclical included types

2008-04-16T20:00:49.0007Z

you have a cyclical relationship:

http://freebase.com/view/schema/user/spatialed/default_domain/satellite_image_product

i click on the first included type "Land cover classification system" of "Satellite Image Product"

http://freebase.com/view/schema/user/spatialed/default_domain/land_cover_classification_system

and "Land cover classification system" has as its first included type "Satellite Image Product"

spatialed: Cyclical included types

2008-04-16T18:30:13.0000Z

i'm seeing a cyclical relationship in:

http://freebase.com/view/schema/user/spatialed/default_domain/satellite_image_product

a satellite image product has itself as an included type.

is this an error?