Jump to content

Blogs

Featured Entries

  • Basil

    Different Types of Sequencing Files

    By Basil


    When dealing with Next-Generation Sequencing data for the first time, you might be a little confused when seeing all the different types of sequencing files that are out there. Although it may seem intimidating at first, a little bit of time around these files and you'll become a sequencing pro in no time!  FASTQ Format FASTQ files are sometimes referred to as the raw sequencing reads. They are usually the format file that you receive from whatever company you have chosen to conduct the Next-Generation Sequencing of your data (or the machine itself, if you performed the sequencing.) The reason we refer to them as raw reads is because the file has all of the reads from your data, without any additional processes conducted on them. The other format files that we talk about later will have had something done to them, as to change the way we can process the data. The image below shows an extremely simplified view of how the FASTQ file comes to be. For example, let's say you are interested in getting heart tissue sequenced for your research. You isolate the heart tissue sample and send it off to a company to get it sequenced. Due to how sequencing is currently conducted by the most popular companies, the file that you will end up getting will be chunks of your original DNA sequence in X amounts of base-pairs (anywhere between 75-200), with a quality score right below the nucleotides. The quality score will be a character that corresponds to a particular number. In our example, we have included the @ quality score, which has a value of 31.         Aligned Format Files: BAM and SAM Raw sequencing files can give you an idea of the quality of the sequencing that was conducted and other general information about your data. But what if you wanted to find out how your heart tissue data was different than the tissue of other individuals? You would not be able to find this information out by just analyzing your raw FASTQ file. This is where genomic alignment comes into play. Genomic alignment is the process of taking your raw sequencing data and aligning it to a reference genome.  (If you don't know what a reference genome, it's an assembled genome sequence that is representative of a particular species.) The SAM file, which stands for sequence aligned mapping file, will have all the reads of your data, just like the FASTQ file had, but it will also have what the reference genome at that particular nucleotide is, right below it.  So, going back to our example data, if we had aligned it to a reference genome, we may see something like this:    As you can see, all of our data matches the references, besides the bolded G. So what does this mean? It could be that at that position, our data has a single nucleotide polymorphism or it could be some sequencing error.  Variant Call Format Files: VCF
    We just mentioned, that comparing our data to a reference genome is useful in finding how our data is different than what the consensus genomic sequence is. At this stage, you could use a genomic viewer, such as the Integrative Genomic Viewer and manually analyze these differences. Or, you could run something called variant-calling, and produce a list of all of the variants that are present, in a file format called a Variant Call Format File, or VCF. A VCF file will tell you the exact position of the variant present, what the allele should have been in comparison to the consensus genome (reference allele), and what the allele currently is for your individual (alternative allele.) An example VCF file is shown below:  The first column of a VCF file is chromosomal location. Depending on what reference genome was used for alignment, you may get chromosome number listed similar to the image (with the chromosome abbreviation, chr, and the number of a chromosome), or you may only get the chromosome number.  The second column has the actual location, within the specified chromosome. The third column in our example has a period, but VCF files typically will have a variant identification number, denoted as a SNP id, in this column, which means that this variant has been identified and is listed within various databases. The fourth column is the reference allele that we referred to above, while the fifth column is the alternative allele. The last two columns both contain tidbits of information that we will discuss in a later article. For now, just know that the sixth column refers to a variant quality score, while the seventh column refers to whether that variant passed or failed a statistical test to remove false-positives. 
    • 1 comment
    • 191 views
  • Basil

    Introduction to SQL

    By Basil

    When you think about handling and processing the huge amount of data, what comes to mind? For many, thoughts of utilizing Python coupled with machine learning algorithms arise. What may not initially come to thought is the notion of using SQL instead. You might be scratching your head at this prospect of using SQL in Data Science or Bioinformatics when there are other alternatives. Or, perhaps, you’re not familiar enough with this particular language to jump to a conclusion. The fact of the matter is that SQL is a programming language you should familiarize yourself with if you’re looking to jump into the world of big data. So why would somebody use SQL instead of the many alternatives? Simply put, SQL provides simplicity and robustness that you can seldom find anywhere else. Add to the equation that Data Science careers sometimes require more than handling big data. A big skill set that one can have is the ability to conduct database management on web applications; a feat for SQL and an RDMS (we’ll discuss this a little bit later.) Database Introduction (Introduction to SQL) For a thorough understanding of SQL and its potential role in Data Science, some basics are needed including an introduction to databases. First off, what exactly is a database? For simplicity, a database is just an organized collection of data. Within this collection, we have even more organization in the form of tables. Tables have specific bits of information stored inside them, and within these tables, there are individual columns that have even more specificity to them. All of this may seem a little confusing, so we’ll go ahead and see a table, called “table1,” inside a database to clear things up. (Word of caution:  The database table and associated column names were created with simplicity in mind. You’ll probably never run across a table titled “table1,” or a column titled “address,” especially when dealing with large databases.)  This sample table contains some information about a fabricated client base. In our actual database, there will be other tables that contain more relevant information, but let’s pretend this is all we need for the moment. As we have mentioned, each column contains a particular characteristic and here we can see the values of clientID, address, city and state. Every single row in our table contains specific data (in this case, a particular client), whereas the columns include universal values or traits. SQL is the language, while a Database Management System (DBMS) is the software that contains and manages the data. Something that a lot of people get confused with is when they hear things about MySQL, SQLite or NoSQL and don’t quite understand its relation to SQL. In our example, we showcased our database table inside a simple Excel file. In a real world example, your data will more than likely be stored in some other software dedicated to database management. This idea is what we refer to as Database Management Software, or DBMS for short. MySQL, SQLite, and NoSQL are all examples of DBMS. You should not worry about mastering DBMS’ until you get the hang of SQL itself. Most DBMS for SQL follow the same protocols, with some minor changes that you can learn later. A particular kind of DBMS named a Relational Database Management System (RDMS) and uses a specific type of modeling called a relational model. The RDMS, in particular, is called MySQL and is a popular database choice for websites. In fact, it’s what our site uses for database management. If you are lost with some of the technical jargon, just remember: SQL is the programming language, and a DBMS is the database system we will be using to manage our data.  Where Can I Practice My Code? Since we aren’t going to go in-depth with the DBMS, you’re probably wondering how you’ll be playing around with SQL code. There are some programs you can download to do so, or you can utilize this free online SQL interpreter. This website allows you to test out some basic SQL code without having to download more complicated DBMS’. The Almighty Select Statement In any SQL or SQL in Data Science course, the first statement you’ll learn is select. It’s quite simple as to why this is the first statement you hear.  In SQL, the objective is to alter and configure the database to suit your needs. To do this, you must select certain attributes and do something with them. However, just selecting something won’t do much. You’ll need to pick something, tell the system where you are selecting it from and then what you want to do with it. This concept of actually doing something with the data leads us to the from statement. Whenever you utilize select, you’ll almost always use from as well. Take a look at the following: Say, for example; you want to choose all of the client’s addresses from the table example from above.  You would utilize the following SQL statement (given that the table is named table1): Select address From table1; The results would yield you all of the addresses for every single row. You could use the same structure if you wanted to pull all of the ClientID’s or any of the other columns from the table. Don’t Forget The Semi-Colon The semi-colon in SQL denotes termination, so you’ll need to place it at the end of your SQL statement. The Where Clause Makes Things Happen While the select and from statements allow you to pick which specific data you want to handle, the where statement allows you to conduct the altering itself. There are numerous operators that the where clause can use to help you make things happen. Using the where clause for equality, non-equality, and showing greater than or less than.  (=, <>, >, <.) Using the where clause to join other clauses. Using the where clause for mathematical calculations. Using the where clause to group data. The list goes on and on for how you can incorporate where into SQL, but just remember that it allows you to make things happen within SQL. Taking the Basics and Applying it to Data Our introduction to SQL was overly simplified and was aimed at providing a very brief introduction to those who have never used SQL. If you’re eager to learn more, we recommend a free MOOC provided by Stanford Laguna.  I’ve personally taken this course, and it’s an excellent introduction to SQL! We touched at the beginning of this article about some of the ways you might be using SQL in Data Science or Bioinformatics careers. Ultimately, it does depend on the particular job. You may never touch SQL once you are in your future career. Or, on the contrary, you may find that your position uses SQL extensively alongside other programming languages and software.  Due to the unexpected nature of whether or not you’re going to need SQL, it’s worth a shot to know it at some level. At the minimum, I would recommend you at least have a basic understanding of SQL and how to do simple database analysis and alterations. Again, you never know if this can come in handy. Some careers may also not care about the specific programming languages, as long as you can conduct the data analysis that they need. Whether you’re using Python, Perl or SQL doesn’t matter nearly as much as whether or not you can perform the tasks.  Final Thoughts If you have used any other SQL MOOC’s or have any useful materials for beginners to grasp this programming language, feel free to post the resources below! Be sure also to join us over at the YDSOA Community Forums as we discuss a wide variety of related topics!
    • 2 comments
    • 334 views

Our community blogs

  1.  

    Scikit-ribo

     

    A New open-source software has been developed by researchers from Cold Spring Harbor Laboratory and John Hopkins University, to address the need for more accurate measurement of protein translation.  Named Scikit-ribo, the tool enables accurate aminoacyl site prediction and estimates of translational efficiency from either Ribo-seq or RNASeq data.

    The software can be downloaded from the Scikit-bibo Github page, https://github.com/hanfang/scikit-ribo. Additionally, Scikit-ribo was published in the bioRxiv preprint server at the following link: http://www.biorxiv.org/content/early/2017/06/27/156588

  2. PathwayMapper

    PathwayMapper

    Although there are many visualization tools available, that allow researchers to explore and analyze their cancer datasets, not many of them provide simplified diagrams. To answer the need for simplification, researchers from Bilkent University, Cornell University, and Oregon Health and Science University have a developed a new online tool. Named PathwayMapper, the web editor allows researchers to create more clear pathways and diagrams, similar to those found in The Cancer Genome Atlas.

    The online tool can be accessed via the PathwayMapper website (www.pathwaymapper.org) or can be downloaded from the PathwayMapper's Github Page. 

    Our Interaction with PathwayMapper

    Our brief interaction with the online web version of PathwayMapper showed us just how easy it was to develop sophisticated pathways while keeping things easy to follow and understand. Users can quickly select different node palettes such as the particular gene, complex, family, etc. Another category allows you to create the various interactions that are occurring between the complexes. Additionally, the software allows you to create customizations to the layout, that gives you the most control over the entire design of the pathway. 

     

  3. Bioinformatics and Computational Biology are two extremely related fields, and as such, many people in the scientific and academic worlds will refer to these two areas interchangeably. But is this truly accurate? Are these fields so close together, that we can lump them into one big category, or are there significant differences that need to be discussed and understood? In this article, we'll go over the commonalities and differences that these two fields face, and which path would be most suitable for you, depending on your research interests. 

    Difference Between Computational Biology and Bioinformatics: What's the Major Difference? 

    Like we mentioned above, to some people, Computational Biology and Bioinformatics hold no difference. To other, such as Dr. Russ Altman of Standford University, there is a very concrete difference between the two. He believes that Bioinformatics is where you create the tools, software, and algorithms that can be used to handle and work with large biological data systems. Likewise, in his mind, Computational Biology is all about learning and study biology, by using the computational tools and software made by Bioinformaticians. 

    Difference_Between_Computational_Biology_and_Bioinformatics.png

    So, according to Dr. Altman's definition, if you're somebody who primarily enjoys being on the creation sides of things, and wanting to add to the available tools and resources for people to analyze their biological data with, Bioinformatics would be the path for you. But, if you'd rather use existing computational tools to study and understand biology better, then you'd probably want to go towards Computational Biology. 

    Computational Biology vs Bioinformatics Academic Programs

    Luckily for many of you who go the Computational Biology and Bioinformatics academic route, most graduate programs combine these two fields into a one-degree program, which allows you more flexibility in figuring out exactly which niche fits you best. But what happens if you're interested in programs that are either Bioinformatics or Computational Biology instead of both? Which one should you choose? 

    If this is your situation, you shouldn't make a decision based upon the name. Instead, look at the faculty members of each program, and see what kind of research they are doing. You might find that a program that is Bioinformatics-based has a lot of Computational Biology research, and vice versa. By researching faculty, and the type of research that is being conducted at prospective universities, you can get an idea of what kind of research you might be involved in if you choose to go to that specific school. 

    Which One Should You Pursue? 

    Choosing whether you want to become a Bioinformatician or a Computational Biologist comes down to figuring out whether you want to be at the forefront of creating computational software for biology or if you'd rather be using these tools to conduct your research. But just remember, things aren't as cut and dry as we've made it out to be, and you still may see job positions or academic programs use these two terms interchangeably. Instead of relying on the title to help decide where you wnat to go, figure out exactly what your research interests are and which program or position does the best job at utilizing them. 

  4. Basil
    Latest Entry

    By Basil,

    When you think about handling and processing the huge amount of data, what comes to mind? For many, thoughts of utilizing Python coupled with machine learning algorithms arise. What may not initially come to thought is the notion of using SQL instead. You might be scratching your head at this prospect of using SQL in Data Science or Bioinformatics when there are other alternatives. Or, perhaps, you’re not familiar enough with this particular language to jump to a conclusion. The fact of the matter is that SQL is a programming language you should familiarize yourself with if you’re looking to jump into the world of big data.

    So why would somebody use SQL instead of the many alternatives? Simply put, SQL provides simplicity and robustness that you can seldom find anywhere else. Add to the equation that Data Science careers sometimes require more than handling big data. A big skill set that one can have is the ability to conduct database management on web applications; a feat for SQL and an RDMS (we’ll discuss this a little bit later.)

    Database Introduction (Introduction to SQL)

    For a thorough understanding of SQL and its potential role in Data Science, some basics are needed including an introduction to databases. First off, what exactly is a database? For simplicity, a database is just an organized collection of data. Within this collection, we have even more organization in the form of tables. Tables have specific bits of information stored inside them, and within these tables, there are individual columns that have even more specificity to them.

    All of this may seem a little confusing, so we’ll go ahead and see a table, called “table1,” inside a database to clear things up.

    Introduction to SQL

    (Word of caution:  The database table and associated column names were created with simplicity in mind. You’ll probably never run across a table titled “table1,” or a column titled “address,” especially when dealing with large databases.) 

    This sample table contains some information about a fabricated client base. In our actual database, there will be other tables that contain more relevant information, but let’s pretend this is all we need for the moment. As we have mentioned, each column contains a particular characteristic and here we can see the values of clientID, address, city and state.

    Every single row in our table contains specific data (in this case, a particular client), whereas the columns include universal values or traits.

    SQL is the language, while a Database Management System (DBMS) is the software that contains and manages the data.

    Something that a lot of people get confused with is when they hear things about MySQL, SQLite or NoSQL and don’t quite understand its relation to SQL. In our example, we showcased our database table inside a simple Excel file. In a real world example, your data will more than likely be stored in some other software dedicated to database management. This idea is what we refer to as Database Management Software, or DBMS for short. MySQL, SQLite, and NoSQL are all examples of DBMS. You should not worry about mastering DBMS’ until you get the hang of SQL itself. Most DBMS for SQL follow the same protocols, with some minor changes that you can learn later.

    A particular kind of DBMS named a Relational Database Management System (RDMS) and uses a specific type of modeling called a relational model. The RDMS, in particular, is called MySQL and is a popular database choice for websites. In fact, it’s what our site uses for database management.

    If you are lost with some of the technical jargon, just remember: SQL is the programming language, and a DBMS is the database system we will be using to manage our data. 

    Where Can I Practice My Code?

    Since we aren’t going to go in-depth with the DBMS, you’re probably wondering how you’ll be playing around with SQL code. There are some programs you can download to do so, or you can utilize this free online SQL interpreter. This website allows you to test out some basic SQL code without having to download more complicated DBMS’.

    The Almighty Select Statement

    In any SQL or SQL in Data Science course, the first statement you’ll learn is select. It’s quite simple as to why this is the first statement you hear.  In SQL, the objective is to alter and configure the database to suit your needs. To do this, you must select certain attributes and do something with them. However, just selecting something won’t do much. You’ll need to pick something, tell the system where you are selecting it from and then what you want to do with it. This concept of actually doing something with the data leads us to the from statement. Whenever you utilize select, you’ll almost always use from as well. Take a look at the following:

    Say, for example; you want to choose all of the client’s addresses from the table example from above.  You would utilize the following SQL statement (given that the table is named table1):

    Select address From table1;

    The results would yield you all of the addresses for every single row. You could use the same structure if you wanted to pull all of the ClientID’s or any of the other columns from the table.

    Don’t Forget The Semi-Colon

    The semi-colon in SQL denotes termination, so you’ll need to place it at the end of your SQL statement.

    The Where Clause Makes Things Happen

    While the select and from statements allow you to pick which specific data you want to handle, the where statement allows you to conduct the altering itself. There are numerous operators that the where clause can use to help you make things happen.

    1. Using the where clause for equality, non-equality, and showing greater than or less than.  (=, <>, >, <.)
    2. Using the where clause to join other clauses.
    3. Using the where clause for mathematical calculations.
    4. Using the where clause to group data.

    The list goes on and on for how you can incorporate where into SQL, but just remember that it allows you to make things happen within SQL.

    Taking the Basics and Applying it to Data

    Our introduction to SQL was overly simplified and was aimed at providing a very brief introduction to those who have never used SQL. If you’re eager to learn more, we recommend a free MOOC provided by Stanford Laguna.  I’ve personally taken this course, and it’s an excellent introduction to SQL!

    We touched at the beginning of this article about some of the ways you might be using SQL in Data Science or Bioinformatics careers. Ultimately, it does depend on the particular job. You may never touch SQL once you are in your future career. Or, on the contrary, you may find that your position uses SQL extensively alongside other programming languages and software. 

    Due to the unexpected nature of whether or not you’re going to need SQL, it’s worth a shot to know it at some level. At the minimum, I would recommend you at least have a basic understanding of SQL and how to do simple database analysis and alterations. Again, you never know if this can come in handy.

    Some careers may also not care about the specific programming languages, as long as you can conduct the data analysis that they need. Whether you’re using Python, Perl or SQL doesn’t matter nearly as much as whether or not you can perform the tasks. 

    Final Thoughts

    If you have used any other SQL MOOC’s or have any useful materials for beginners to grasp this programming language, feel free to post the resources below! Be sure also to join us over at the YDSOA Community Forums as we discuss a wide variety of related topics!

  5. Graduate Discussion

    • 0
      entries
    • 0
      comments
    • 167
      views

    No blog entries yet

×